[jira] [Commented] (HBASE-4951) master process can not be stopped when it is initializing
[ https://issues.apache.org/jira/browse/HBASE-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165916#comment-13165916 ] xufeng commented on HBASE-4951: --- @ramkrishna thanks. do you think we should fix it in 0.90. I try to create a path. > master process can not be stopped when it is initializing > - > > Key: HBASE-4951 > URL: https://issues.apache.org/jira/browse/HBASE-4951 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.3 >Reporter: xufeng >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.92.0, 0.90.5 > > > It is easy to reproduce by following step: > step1:start master process.(do not start regionserver process in the cluster). > the master will wait the regionserver to check in: > org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) to > checkin > step2:stop the master by sh command bin/hbase master stop > result:the master process will never die because catalogTracker.waitForRoot() > method will block unitl the root region assigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4951) master process can not be stopped when it is initializing
[ https://issues.apache.org/jira/browse/HBASE-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-4951: - Assignee: ramkrishna.s.vasudevan > master process can not be stopped when it is initializing > - > > Key: HBASE-4951 > URL: https://issues.apache.org/jira/browse/HBASE-4951 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.3 >Reporter: xufeng >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 0.92.0, 0.90.5 > > > It is easy to reproduce by following step: > step1:start master process.(do not start regionserver process in the cluster). > the master will wait the regionserver to check in: > org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) to > checkin > step2:stop the master by sh command bin/hbase master stop > result:the master process will never die because catalogTracker.waitForRoot() > method will block unitl the root region assigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4987: --- Attachment: HBASE-4987.D675.3.patch khemani updated the revision "HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager". Reviewers: tedyu, nspiegelberg, dhruba, JIRA narrowed the synchronization block This fixes of two issues 1. double counting of batch.installed is fixed. This is a simple fix. If the new Task(batch) is not used then decrement batch.installed--. No lock is requited to decrement batch.installed. Only one thread ever looks at this value. 2. There was an unhandled race between the zookeeper-callback thread calling setDone() on a task not part of any batch and another thread incorporating that task into a batch. That is now fixed. REVISION DETAIL https://reviews.facebook.net/D675 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Fix For: 0.92.0 > > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch, > HBASE-4987.D675.3.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4951) master process can not be stopped when it is initializing
[ https://issues.apache.org/jira/browse/HBASE-4951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165903#comment-13165903 ] ramkrishna.s.vasudevan commented on HBASE-4951: --- @Xufeng. I used ./hbase-daemon.sh stop master and it stopped. My bad.. if we use ./hbase master stop it doesnt stop. > master process can not be stopped when it is initializing > - > > Key: HBASE-4951 > URL: https://issues.apache.org/jira/browse/HBASE-4951 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.3 >Reporter: xufeng >Priority: Critical > Fix For: 0.92.0, 0.90.5 > > > It is easy to reproduce by following step: > step1:start master process.(do not start regionserver process in the cluster). > the master will wait the regionserver to check in: > org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver(s) to > checkin > step2:stop the master by sh command bin/hbase master stop > result:the master process will never die because catalogTracker.waitForRoot() > method will block unitl the root region assigned. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165898#comment-13165898
]
chunhui shen commented on HBASE-4988:
-
@stack
The case happens in our test environment which use 0.90 version.
If .META. Server is killed and not started immediately.
the step MetaEditor.offlineParentInMeta will fail and throw exception,
and the JournalEntry.PONR causes server abort when rolling back.
In Trunk version, MetaEditor.offlineParentInMeta will retry, but the parent
region can't on service for a long time, I think it is unacceptable. Also the
retry would be failed, and cause server abort finally.
{code}metaServer.put(CatalogTracker.META_REGION_NAME, put);{code}
If the .meta. server die between verification and doing put above, it will
abort because we can't ensure whether update .meta. successfully. However, if
we can find that .meta. server is not ok now first, we needn't abort server
which is doing split
> MetaServer crash cause all splitting regionserver abort
> ---
>
> Key: HBASE-4988
> URL: https://issues.apache.org/jira/browse/HBASE-4988
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
> Attachments: hbase-4988v1.patch
>
>
> If metaserver crash now,
> All the splitting regionserver will abort theirself.
> Becasue the code
> {code}
> this.journal.add(JournalEntry.PONR);
> MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
> this.parent.getRegionInfo(), a.getRegionInfo(),
> b.getRegionInfo());
> {code}
> If the JournalEntry is PONR, split's roll back will abort itselef.
> It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4965) Monitor the open file descriptors and the threads counters during the unit tests
[ https://issues.apache.org/jira/browse/HBASE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165889#comment-13165889 ] stack commented on HBASE-4965: -- All tests pass for you N? I'm seeing a hang soon after we move to medium sized... let me retry. > Monitor the open file descriptors and the threads counters during the unit > tests > > > Key: HBASE-4965 > URL: https://issues.apache.org/jira/browse/HBASE-4965 > Project: HBase > Issue Type: Improvement > Components: test >Affects Versions: 0.94.0 > Environment: all >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 4965_all.patch, ResourceChecker.java, > ResourceCheckerJUnitRule.java > > > We're seeing a lot of issues with hadoop-qa related to threads or file > descriptors. > Monitoring these counters would ease the analysis. > Note as well that > - if we want to execute the tests in the same jvm (because the test is small > or because we want to share the cluster) we can't afford to leak too many > resources > - if the tests leak, it's more difficult to detect a leak in the software > itself. > I attach piece of code that I used. It requires two lines in a unit test > class to: > - before every test, count the threads and the open file descriptor > - after every test, compare with the previous value. > I ran it on some tests; we have for example: > - client.TestMultiParallel#testBatchWithManyColsInOneRowGetAndPut: 232 > threads (was 231), 390 file descriptors (was 390). => TestMultiParallel uses > 232 threads! > - client.TestMultipleTimestamps#testWithColumnDeletes: 152 threads (was 151), > 283 file descriptors (was 282). > - client.TestAdmin#testCheckHBaseAvailableClosesConnection: 477 threads (was > 294), 815 file descriptors (was 461) > - client.TestMetaMigrationRemovingHTD#testMetaMigration: 149 threads (was > 148), 310 file descriptors (was 307). > It's not always leaks, we can expect some pooling effects. But still... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-4969.
--
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: Reviewed
Committed branch and trunk. Thanks for patch Prakash.
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Fix For: 0.92.0
>
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4940) hadoop-metrics.properties can include configuration of the "rest" context for ganglia
[
https://issues.apache.org/jira/browse/HBASE-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165883#comment-13165883
]
stack commented on HBASE-4940:
--
@Mubarak Excellent! (Good luck!)
> hadoop-metrics.properties can include configuration of the "rest" context for
> ganglia
> -
>
> Key: HBASE-4940
> URL: https://issues.apache.org/jira/browse/HBASE-4940
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 0.90.5
> Environment: HBase-0.90.1
>Reporter: Mubarak Seyed
>Priority: Minor
> Labels: hbase-rest
> Fix For: 0.90.5
>
>
> It appears from hadoop-metrics.properties that configuration for rest context
> is missing. It would be good if we add the rest context and commented out
> them, if anyone is using rest-server and if they want to monitor using
> ganglia context then they can uncomment the rest context and use them for
> rest-server monitoring using ganglia.
> {code}
> # Configuration of the "rest" context for ganglia
> #rest.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> #rest.period=10
> #rest.servers=ganglia-metad-hostname:port
> {code}
> Working on the patch, will submit it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4987: - Resolution: Fixed Fix Version/s: 0.92.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed branch and trunk. Thanks for the patch Prakash. I didn't do the right arc thing because still waiting on a little input on problem over in a previous issue trying to do arc (I'll figure it one of these days). > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Fix For: 0.92.0 > > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4880:
-
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed branch and trunk. Nice work Chunhui.
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0
>
> Attachments: 4880-0.92.txt, 4880.txt, hbase-4880.patch,
> hbase-4880v2.patch, hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165876#comment-13165876 ] Ted Yu commented on HBASE-4120: --- In PriorityFunction.java, several lines are much wider than 80 characters. Please use the formatter from HBASE-3678 on the new files. > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, > TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, > TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4880:
-
Attachment: 4880-0.92.txt
0.92 patch is a little different; e.g. no test categories in branch.
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0
>
> Attachments: 4880-0.92.txt, 4880.txt, hbase-4880.patch,
> hbase-4880v2.patch, hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165873#comment-13165873 ] stack commented on HBASE-4352: -- It seemed like 5d0a564b65a53b1c86f7c02407c82231 was opened fine (I'd just done clean cluster start, killed master, started 0.90.5 master, then had tried the rolling restart --- no load on cluster). I didn't dig in too deep. Was going to start adding logging... debugging... but had spent enough time on it. Can I roll a 0.90.5RC Ram? > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165872#comment-13165872 ] ramkrishna.s.vasudevan commented on HBASE-4352: --- @Stack 5d0a564b65a53b1c86f7c02407c82231 - Did this region get opened up correctly? Then the second set of logs is not a problem. First set of logs am not very clear as why it came up. Thanks Stack for your time.:) > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165870#comment-13165870
]
Ted Yu commented on HBASE-4946:
---
Integrated to 0.92 and TRUNK.
Thanks for the patch Andrei.
Thanks for the review Lars and Stack.
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If yo
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165869#comment-13165869
]
stack commented on HBASE-4988:
--
Which hbase version Chunhui? In 0.92 we should be retrying (the putToMetaTable
should be retrying and will give up only if .META. really gond)? This change
has no retry facility and would seem to be a regression?
{code}
+metaServer.put(CatalogTracker.META_REGION_NAME, put);
{code}
The .meta. server could die between verification and your doing above.
Maybe have putToMetaTable return true/false?
Also, I don't think this will work:
{code}
+toPONR = MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
+this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
{code}
In the past, I've seen weird case where the offlineParentInMeta looked like it
failed but all that happened was that the client timeout trying to do the
.META. put -- the edit actually went in anyways. Aborting the regionserver is
pretty radical but having the PONR before we do the meta edit would at least
make it so if the edit went in, we'd likely find out during server recovery.
We used to have PONR after the above but this patch changed it: HBASE-4562
You have a point though that currently if .META. goes away long enough, we
could abort all servers (this is 0.92?). This is pretty serious issue.
> MetaServer crash cause all splitting regionserver abort
> ---
>
> Key: HBASE-4988
> URL: https://issues.apache.org/jira/browse/HBASE-4988
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
> Attachments: hbase-4988v1.patch
>
>
> If metaserver crash now,
> All the splitting regionserver will abort theirself.
> Becasue the code
> {code}
> this.journal.add(JournalEntry.PONR);
> MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
> this.parent.getRegionInfo(), a.getRegionInfo(),
> b.getRegionInfo());
> {code}
> If the JournalEntry is PONR, split's roll back will abort itselef.
> It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165867#comment-13165867
]
Hadoop QA commented on HBASE-4946:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506703/4946-v5.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/479//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/479//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/479//console
This message is automatically generated.
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doP
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165864#comment-13165864
]
Hudson commented on HBASE-4980:
---
Integrated in HBase-0.92 #179 (See
[https://builds.apache.org/job/HBase-0.92/179/])
HBASE-4980 Null pointer exception in HBaseClient receiveResponse
HBASE-4980 Null pointer exception in HBaseClient receiveResponse
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
stack :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
>Assignee: Shrijeet Paliwal
> Labels: newbie
> Fix For: 0.92.0
>
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4946:
--
Fix Version/s: 0.94.0
0.92.0
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/Con
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165863#comment-13165863
]
stack commented on HBASE-4946:
--
This belongs in 0.92 too?
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/Co
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165861#comment-13165861 ] Hadoop QA commented on HBASE-4987: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506701/HBASE-4987.D675.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/478//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/478//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/478//console This message is automatically generated. > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504702/TablePriority_v8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -143 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 88 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.master.TestRestartCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/328//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/328//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/328//console This message is automatically generated.) > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, > TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, > TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502140/TablePriority_v8_for_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -145 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary org.apache.hadoop.hbase.master.TestDistributedLogSplitting Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/154//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/154//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/154//console This message is automatically generated.) > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, > TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, > TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4120: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12504706/TablePriority_v8.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -143 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 88 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRestartCluster org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/330//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/330//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/330//console This message is automatically generated.) > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, > TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, > TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165857#comment-13165857
]
Ted Yu commented on HBASE-4120:
---
Please add license to QosRegionObserver.java
I think we should enhance ScannerListener.leaseExpired() with
pre/postScannerClose() so that features such as table priority can receive
consistent notification.
The trick here is that RegionScanner only exposes HRegionInfo. We should be
able to utilize onlineRegions in looking up HRegion by region name.
Then we should be able to call the following:
{code}
if (region != null && region.getCoprocessorHost() != null) {
region.getCoprocessorHost().postScannerClose(s);
}
{code}
> isolation and allocation
>
>
> Key: HBASE-4120
> URL: https://issues.apache.org/jira/browse/HBASE-4120
> Project: HBase
> Issue Type: New Feature
> Components: master, regionserver
>Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0
>Reporter: Liu Jia
>Assignee: Liu Jia
> Fix For: 0.94.0
>
> Attachments: Design_document_for_HBase_isolation_and_allocation.pdf,
> Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
> HBase_isolation_and_allocation_user_guide.pdf,
> Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch,
> TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch,
> TablePriority_v8.patch, TablePriority_v8_for_trunk.patch,
> TablePrioriy_v9.patch
>
>
> The HBase isolation and allocation tool is designed to help users manage
> cluster resource among different application and tables.
> When we have a large scale of HBase cluster with many applications running on
> it, there will be lots of problems. In Taobao there is a cluster for many
> departments to test their applications performance, these applications are
> based on HBase. With one cluster which has 12 servers, there will be only one
> application running exclusively on this server, and many other applications
> must wait until the previous test finished.
> After we add allocation manage function to the cluster, applications can
> share the cluster and run concurrently. Also if the Test Engineer wants to
> make sure there is no interference, he/she can move out other tables from
> this group.
> In groups we use table priority to allocate resource, when system is busy; we
> can make sure high-priority tables are not affected lower-priority tables
> Different groups can have different region server configurations, some groups
> optimized for reading can have large block cache size, and others optimized
> for writing can have large memstore size.
> Tables and region servers can be moved easily between groups; after changing
> the configuration, a group can be restarted alone instead of restarting the
> whole cluster.
> git entry : https://github.com/ICT-Ope/HBase_allocation .
> We hope our work is helpful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165853#comment-13165853 ] Phabricator commented on HBASE-4987: stack has commented on the revision "HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager". +1 REVISION DETAIL https://reviews.facebook.net/D675 > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165851#comment-13165851
]
Lars Hofhansl commented on HBASE-4946:
--
+1 v5 lgtm
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/C
[jira] [Updated] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to des
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4946:
--
Attachment: 4946-v5.txt
Patch v5 removes commented out code.
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: 4946-v4.txt, 4946-v5.txt, HBASE-4946-v2.patch,
> HBASE-4946-v3.patch, HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministra
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165841#comment-13165841
]
Lars Hofhansl commented on HBASE-4946:
--
Please don't comment out code, but remove it instead. We have a record of the
old code still in source control.
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: 4946-v4.txt, HBASE-4946-v2.patch, HBASE-4946-v3.patch,
> HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incor
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165842#comment-13165842 ] Phabricator commented on HBASE-4987: tedyu has commented on the revision "HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager". The new patch relies on oldtask.deleted to not double increment batch.installed Interesting. REVISION DETAIL https://reviews.facebook.net/D675 > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4987: -- Status: Patch Available (was: Open) > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4946) HTable.coprocessorExec (and possibly coprocessorProxy) does not work with dynamically loaded coprocessors (from hdfs or local system), because the RPC system tries to d
[
https://issues.apache.org/jira/browse/HBASE-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165839#comment-13165839
]
Ted Yu commented on HBASE-4946:
---
Will commit patch v4 tomorrow.
> HTable.coprocessorExec (and possibly coprocessorProxy) does not work with
> dynamically loaded coprocessors (from hdfs or local system), because the RPC
> system tries to deserialize an unknown class.
> -
>
> Key: HBASE-4946
> URL: https://issues.apache.org/jira/browse/HBASE-4946
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.0
>Reporter: Andrei Dragomir
>Assignee: Andrei Dragomir
> Attachments: 4946-v4.txt, HBASE-4946-v2.patch, HBASE-4946-v3.patch,
> HBASE-4946.patch
>
>
> Loading coprocessors jars from hdfs works fine. I load it from the shell,
> after setting the attribute, and it gets loaded:
> {noformat}
> INFO org.apache.hadoop.hbase.regionserver.HRegion: Setting up tabledescriptor
> config now ...
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: Class
> com.MyCoprocessorClass needs to be loaded from a file -
> hdfs://localhost:9000/coproc/rt- >0.0.1-SNAPSHOT.jar.
> INFO org.apache.hadoop.hbase.coprocessor.CoprocessorHost: loadInstance:
> com.MyCoprocessorClass
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost:
> RegionEnvironment createEnvironment
> DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Registered protocol
> handler: region=t1,,1322572939753.6409aee1726d31f5e5671a59fe6e384f.
> protocol=com.MyCoprocessorClassProtocol
> INFO org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost: Load
> coprocessor com.MyCoprocessorClass from HTD of t1 successfully.
> {noformat}
> The problem is that this coprocessors simply extends BaseEndpointCoprocessor,
> with a dynamic method. When calling this method from the client with
> HTable.coprocessorExec, I get errors on the HRegionServer, because the call
> cannot be deserialized from writables.
> The problem is that Exec tries to do an "early" resolve of the coprocessor
> class. The coprocessor class is loaded, but it is in the context of the
> HRegionServer / HRegion. So, the call fails:
> {noformat}
> 2011-12-02 00:34:17,348 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable:
> Error in readFields
> java.io.IOException: Protocol class com.MyCoprocessorClassProtocol not found
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:125)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:575)
> at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:105)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1237)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1167)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException: com.MyCoprocessorClassProtocol
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
> at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:122)
> ... 10 more
> {noformat}
> Probably the correct way to fix this is to make Exec really smart, so that it
> knows all the class definitions loaded in CoprocessorHost(s).
> I created a small patch that simply doesn't resolve the class definition in
> the Exec, instead passing it as string down to the HRegion layer. This layer
> knows all the definitions, and simply loads it by name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactA
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165837#comment-13165837
]
Ted Yu commented on HBASE-4880:
---
+1 on patch v4.
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
> hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[
https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165828#comment-13165828
]
stack commented on HBASE-4352:
--
Here is a 0.90.5 master trying to reload a regionserver that has been restarted
as a 0.90.5 (it had been a 0.90.4):
{code}
2011-12-08 22:56:53,807 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Assigning region
TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231. to
sv4r9s38,7003,1323384867368
2011-12-08 22:56:53,809 WARN org.apache.hadoop.ipc.HBaseClient: Unexpected
exception receiving call responses
java.lang.NullPointerException
2011-12-08 22:56:53,810 WARN org.apache.hadoop.hbase.master.AssignmentManager:
Failed assignment of
TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231. to
serverName=sv4r9s38,7003,1323384867368, load=(requests=0, regions=0,
usedHeap=23, maxHeap=493), trying to assign elsewhere instead; retry=0
java.io.IOException: Call to /10.4.9.38:7003 failed on local exception:
java.io.IOException: Unexpected exception receiving call responses
at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:806)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:775)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy7.openRegion(Unknown Source)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:561)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1043)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:858)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:838)
at
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:92)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:156)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Unexpected exception receiving call responses
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:481)
Caused by: java.lang.NullPointerException
{code}
It looks like call is null as per another issue that just got committed.
Shouldn't be.
Was getting bunches of this on regionserver side, though it don't seem directly
related:
{code}
2011-12-08 22:56:53,809 DEBUG
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Processing open
of TestTable,0013681953,1323325588029.5d0a564b65a53b1c86f7c02407c82231.
2011-12-08 22:56:53,810 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:7003-0x1341fa8d95400cc Attempting to transition node
5d0a564b65a53b1c86f7c02407c82231 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2011-12-08 22:56:53,811 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:7003-0x1341fa8d95400cc Attempt to transition the unassigned node
for 5d0a564b65a53b1c86f7c02407c82231 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING failed, the node existed but was version 2 not the
expected version -1
2011-12-08 22:56:53,811 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed
transition from OFFLINE to OPENING for region=5d0a564b65a53b1c86f7c02407c82231
2011-12-08 22:56:53,811 WARN
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Region was
hijacked? It no longer exists, encodedName=5d0a564b65a53b1c86f7c02407c82231
{code}
I was hoping to put up a 0.90.5RC. If you want me to hold a while, I can.
Else, I'll go ahead and cut an RC w/o this.
> Apply version of hbase-4015 to branch
> -
>
> Key: HBASE-4352
> URL: https://issues.apache.org/jira/browse/HBASE-4352
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: ramkrishna.s.vasudevan
>Priority: Blocker
> Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch
>
>
> Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface
> so would need move change to end of the Interface and then test that it
> doesn't break rolling restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165811#comment-13165811
]
chunhui shen commented on HBASE-4988:
-
In patchv1,
when doing MetaEditor.offlineParentInMeta, if metaserver is not ok now ,
doesn't add JournalEntry.PONR.
Therefore, rollback won't abort itself
> MetaServer crash cause all splitting regionserver abort
> ---
>
> Key: HBASE-4988
> URL: https://issues.apache.org/jira/browse/HBASE-4988
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
> Attachments: hbase-4988v1.patch
>
>
> If metaserver crash now,
> All the splitting regionserver will abort theirself.
> Becasue the code
> {code}
> this.journal.add(JournalEntry.PONR);
> MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
> this.parent.getRegionInfo(), a.getRegionInfo(),
> b.getRegionInfo());
> {code}
> If the JournalEntry is PONR, split's roll back will abort itselef.
> It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-4988:
Attachment: hbase-4988v1.patch
> MetaServer crash cause all splitting regionserver abort
> ---
>
> Key: HBASE-4988
> URL: https://issues.apache.org/jira/browse/HBASE-4988
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
> Attachments: hbase-4988v1.patch
>
>
> If metaserver crash now,
> All the splitting regionserver will abort theirself.
> Becasue the code
> {code}
> this.journal.add(JournalEntry.PONR);
> MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
> this.parent.getRegionInfo(), a.getRegionInfo(),
> b.getRegionInfo());
> {code}
> If the JournalEntry is PONR, split's roll back will abort itselef.
> It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4987: --- Attachment: HBASE-4987.D675.2.patch khemani updated the revision "HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager". Reviewers: tedyu, nspiegelberg, dhruba, JIRA synchronization on task is needed REVISION DETAIL https://reviews.facebook.net/D675 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch, HBASE-4987.D675.2.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
MetaServer crash cause all splitting regionserver abort
---
Key: HBASE-4988
URL: https://issues.apache.org/jira/browse/HBASE-4988
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
If metaserver crash now,
All the splitting regionserver will abort theirself.
Becasue the code
{code}
this.journal.add(JournalEntry.PONR);
MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
this.parent.getRegionInfo(), a.getRegionInfo(), b.getRegionInfo());
{code}
If the JournalEntry is PONR, split's roll back will abort itselef.
It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4988) MetaServer crash cause all splitting regionserver abort
[
https://issues.apache.org/jira/browse/HBASE-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165805#comment-13165805
]
chunhui shen commented on HBASE-4988:
-
logs
{code}
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined
writetest,28TPVACCO3EI47TH472E1997TX1ZDFQ7XUCMBA2LUKOD7G0U3NQ2L2FG0ILRGZ5ETHFESE5QIMFN8ONUDUXB80G7MEK58G7YM4EG,1323251351741.6399c204b8d45568a782fd0157d6700d.;
next sequenceid=3483318538
2011-12-07 17:49:17,737 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for
writetest,2FB\xC0EE\xC2LDFG\xC8\xB6GV\xCE\xC6F4<\xBBE\xC87BM\xC0\xCD\xC3\xC8A\xB3\xCE\xD5G\xCBI\xBA\xBB\xCB\xD7R\xD2=\xC5>2U;P\xD2D\xCD\xBA\xC6\xC6A\xC1KI\xCDND\xC8\xCEKG\xC3\xCC\xCD\xB4\xC1=\xD0\xC4\xD2FSSPE\xD0V\xCE5@\xBCCN\xC4\xCB\xBE7L\xC8E\xC1\xBD\xCFH,1323251351741.a639e2eda8b2de9ca368c1a13ebbcb44.
because Region has references on open; priority=16, compaction queue size=1
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Failed verification of .META.,,1 at address=dw83.kgb.sqa.cm4:60020;
java.io.EOFException
2011-12-07 17:49:17,737 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Current cached META location is not valid, resetting
2011-12-07 17:49:17,740 INFO org.apache.hadoop.hbase.catalog.CatalogTracker:
Failed verification of .META.,,1 at address=dw83.kgb.sqa.cm4:60020;
java.net.ConnectException: Connection refused
2011-12-07 17:49:17,740 WARN
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Running rollback of
failed split of
writetest,28TPVACCO3EI47TH472E1997TX1ZDFQ7XUCMBA2LUKOD7G0U3NQ2L2FG0ILRGZ5ETHFESE5QIMFN8ONUDUXB80G7MEK58G7YM4EG,1323240352298.c7bde4437e5b12bc7226485dcbc2700b.;
Timed out (2147483647ms)
2011-12-07 17:49:17,740 FATAL
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
serverName=dw87.kgb.sqa.cm4,60020,1323244700069, load=(requests=393,
regions=12, usedHeap=742, maxHeap=15872): Abort; we got an error after
point-of-no-return
{code}
> MetaServer crash cause all splitting regionserver abort
> ---
>
> Key: HBASE-4988
> URL: https://issues.apache.org/jira/browse/HBASE-4988
> Project: HBase
> Issue Type: Bug
>Reporter: chunhui shen
>
> If metaserver crash now,
> All the splitting regionserver will abort theirself.
> Becasue the code
> {code}
> this.journal.add(JournalEntry.PONR);
> MetaEditor.offlineParentInMeta(server.getCatalogTracker(),
> this.parent.getRegionInfo(), a.getRegionInfo(),
> b.getRegionInfo());
> {code}
> If the JournalEntry is PONR, split's roll back will abort itselef.
> It is terrible in huge putting environment when metaserver crash
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165802#comment-13165802 ] gaojinchao commented on HBASE-4970: --- I also want to add a parameter to change keepAliveTime of Htable thread pool. so that clients can have more option > Allow better control of resource consumption in HTable (backport HBASE-4805 > to 0.90 branch) > --- > > Key: HBASE-4970 > URL: https://issues.apache.org/jira/browse/HBASE-4970 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.90.4 >Reporter: gaojinchao >Assignee: gaojinchao >Priority: Trivial > Fix For: 0.90.5 > > Attachments: HBASE-4970_Branch90.patch, > HBASE-4970_Branch90_V1_trial.patch > > > In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES > is slowed down. > Why increasing keepAliveTime of HBase thread pool is slowing down our problem > occurance [RES value increase]? > You can go through the source of sun.nio.ch.Util. Every thread hold 3 > softreference of direct buffer(mustangsrc) for reusage. The code names the 3 > softreferences buffercache. If the buffer was all occupied or none was > suitable in size, and new request comes, new direct buffer is allocated. > After the service, the bigger one replaces the smaller one in buffercache. > The replaced buffer is released. > So I think we can add a parameter to change keepAliveTime of Htable thread > pool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4987: --- Attachment: HBASE-4987.D675.1.patch khemani requested code review of "HBASE-4987 [jira] wrong use of incarnation var in SplitLogManager". Reviewers: tedyu, nspiegelberg, dhruba, JIRA redo HBASE-4855 TEST PLAN TestSplitLogManager REVISION DETAIL https://reviews.facebook.net/D675 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/1485/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > Attachments: HBASE-4987.D675.1.patch > > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165783#comment-13165783
]
ramkrishna.s.vasudevan commented on HBASE-4880:
---
+1 on patch.
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
> hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165779#comment-13165779
]
Hadoop QA commented on HBASE-4880:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506692/hbase-4880v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 74 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/477//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/477//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/477//console
This message is automatically generated.
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
> hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165767#comment-13165767 ] Hudson commented on HBASE-4982: --- Integrated in HBase-0.92 #178 (See [https://builds.apache.org/job/HBase-0.92/178/]) HBASE-4982 graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/bin/graceful_stop.sh > graceful_stop.sh does not pass on the --config its passed to its internal > invocations of other hbase scripts > > > Key: HBASE-4982 > URL: https://issues.apache.org/jira/browse/HBASE-4982 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: stack >Assignee: stack > Fix For: 0.90.5 > > Attachments: 4982.txt > > > Means, unless conf is in default location, we mess up asking zk for state > (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4987) wrong use of incarnation var in SplitLogManager
[ https://issues.apache.org/jira/browse/HBASE-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165766#comment-13165766 ] Prakash Khemani commented on HBASE-4987: Old issue HBASE-4855 > wrong use of incarnation var in SplitLogManager > --- > > Key: HBASE-4987 > URL: https://issues.apache.org/jira/browse/HBASE-4987 > Project: HBase > Issue Type: Bug >Reporter: Prakash Khemani >Assignee: Prakash Khemani > > @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't > think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4987) wrong use of incarnation var in SplitLogManager
wrong use of incarnation var in SplitLogManager --- Key: HBASE-4987 URL: https://issues.apache.org/jira/browse/HBASE-4987 Project: HBase Issue Type: Bug Reporter: Prakash Khemani Assignee: Prakash Khemani @Ramakrishna found and analyzed an issue in SplitLogManager. But I don't think that the fix is correct. Will upload a patch shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165737#comment-13165737
]
chunhui shen commented on HBASE-4880:
-
Address Ram and Stack comments in patchv4
Thanks.
Please check
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
> hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4880) Region is on service before openRegionHandler completes, may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-4880:
Attachment: hbase-4880v4.patch
> Region is on service before openRegionHandler completes, may cause data loss
>
>
> Key: HBASE-4880
> URL: https://issues.apache.org/jira/browse/HBASE-4880
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0, 0.94.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: 4880.txt, hbase-4880.patch, hbase-4880v2.patch,
> hbase-4880v3.patch, hbase-4880v4.patch
>
>
> OpenRegionHandler in regionserver is processed as the following steps:
> {code}
> 1.openregion()(Through it, closed = false, closing = false)
> 2.addToOnlineRegions(region)
> 3.update .meta. table
> 4.update ZK's node state to RS_ZK_REGION_OPEND
> {code}
> We can find that region is on service before Step 4.
> It means client could put data to this region after step 3.
> What will happen if step 4 is failed processing?
> It will execute OpenRegionHandler#cleanupFailedOpen which will do closing
> region, and master assign this region to another regionserver.
> If closing region is failed, the data which is put between step 3 and step 4
> may loss, because the region has been opend on another regionserver and be
> put new data. Therefore, it may not be recoverd through replayRecoveredEdit()
> because the edit's LogSeqId is smaller than current region SeqId.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165724#comment-13165724 ] ramkrishna.s.vasudevan commented on HBASE-4352: --- @Stack Where is the NPE coming in the client side? Could you paste the stack trace to chk it out. > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4352: -- Reopened after reverting patch from 0.90 branch. > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4352: - Fix Version/s: (was: 0.90.5) > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165718#comment-13165718 ] stack commented on HBASE-4352: -- Backing this out. Can't do rolling restart from 0.90.4 to 0.90.5. I get NPEs client-side trying to receive response even after putting up a 0.90.5 master first and while trying to rolling restart a 0.90.4 to 0.90.5 regionserver; the offloading works but then the onloading fails. Without this patch I can rolling restart from 0.90.4 to 0.90.5 smoothly. > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 0.90.5 > > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165706#comment-13165706 ] Hudson commented on HBASE-4982: --- Integrated in HBase-TRUNK #2531 (See [https://builds.apache.org/job/HBase-TRUNK/2531/]) HBASE-4982 graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts stack : Files : * /hbase/trunk/bin/graceful_stop.sh > graceful_stop.sh does not pass on the --config its passed to its internal > invocations of other hbase scripts > > > Key: HBASE-4982 > URL: https://issues.apache.org/jira/browse/HBASE-4982 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: stack >Assignee: stack > Fix For: 0.90.5 > > Attachments: 4982.txt > > > Means, unless conf is in default location, we mess up asking zk for state > (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165695#comment-13165695 ] stack commented on HBASE-4981: -- duh > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165694#comment-13165694 ] Lars Hofhansl commented on HBASE-4981: -- bq. This is sweet. I like passing attribute at same time. Then could have RAW => true and DEBUG => true. Issue I was talking of was HBASE-4981, a debug filter. Actually HBASE-4981 is this one :)... You mean HBASE-4926? > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165679#comment-13165679 ] Lars Hofhansl commented on HBASE-4682: -- Thanks for the review stack (I added comments to the pieces in Import) > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4981: - Attachment: 4981-v3.txt Final change that was committed. > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-4981. -- Resolution: Fixed Hadoop Flags: Reviewed Thanks for the review stack. > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt, 4981-v3.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4965) Monitor the open file descriptors and the threads counters during the unit tests
[
https://issues.apache.org/jira/browse/HBASE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165639#comment-13165639
]
stack commented on HBASE-4965:
--
On hadoop-qa being set to 1024 fds only, thats weird. We dump the ulimit
before the test starts and it shows:
{code}
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 2048
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
6
Running in Jenkins mode
{code}
... so 60k.
So, I wonder where disconnect between your finding and ulimit is? We're
running as a different user after ulimit is output?
I love that leaks report. Thats excellent.
Trying the patch locally
> Monitor the open file descriptors and the threads counters during the unit
> tests
>
>
> Key: HBASE-4965
> URL: https://issues.apache.org/jira/browse/HBASE-4965
> Project: HBase
> Issue Type: Improvement
> Components: test
>Affects Versions: 0.94.0
> Environment: all
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Attachments: 4965_all.patch, ResourceChecker.java,
> ResourceCheckerJUnitRule.java
>
>
> We're seeing a lot of issues with hadoop-qa related to threads or file
> descriptors.
> Monitoring these counters would ease the analysis.
> Note as well that
> - if we want to execute the tests in the same jvm (because the test is small
> or because we want to share the cluster) we can't afford to leak too many
> resources
> - if the tests leak, it's more difficult to detect a leak in the software
> itself.
> I attach piece of code that I used. It requires two lines in a unit test
> class to:
> - before every test, count the threads and the open file descriptor
> - after every test, compare with the previous value.
> I ran it on some tests; we have for example:
> - client.TestMultiParallel#testBatchWithManyColsInOneRowGetAndPut: 232
> threads (was 231), 390 file descriptors (was 390). => TestMultiParallel uses
> 232 threads!
> - client.TestMultipleTimestamps#testWithColumnDeletes: 152 threads (was 151),
> 283 file descriptors (was 282).
> - client.TestAdmin#testCheckHBaseAvailableClosesConnection: 477 threads (was
> 294), 815 file descriptors (was 461)
> - client.TestMetaMigrationRemovingHTD#testMetaMigration: 149 threads (was
> 148), 310 file descriptors (was 307).
> It's not always leaks, we can expect some pooling effects. But still...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4980:
-
Resolution: Fixed
Fix Version/s: 0.92.0
Assignee: Shrijeet Paliwal
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed trunk and branch. Thanks for the patch Shrjeet.
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
>Assignee: Shrijeet Paliwal
> Labels: newbie
> Fix For: 0.92.0
>
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165631#comment-13165631 ] Hadoop QA commented on HBASE-4682: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506658/4682-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/476//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/476//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/476//console This message is automatically generated. > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4973) On failure, HBaseAdmin sleeps one time too many
[
https://issues.apache.org/jira/browse/HBASE-4973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4973:
-
Resolution: Fixed
Fix Version/s: 0.94.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed TRUNK. Thanks for patch N.
> On failure, HBaseAdmin sleeps one time too many
> ---
>
> Key: HBASE-4973
> URL: https://issues.apache.org/jira/browse/HBASE-4973
> Project: HBase
> Issue Type: Improvement
> Components: client
>Affects Versions: 0.94.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4973_HBaseAdmin.patch
>
>
> In this code last sleep is useless as we're not retrying. This can slow down
> failure scenarios by a few seconds (up to 32 second).
> {noformat}
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier = this.conf.getInt(
> "hbase.client.retries.longer.multiplier", 10);
> int tries = 0;
> for (; tries < numRetries; ++tries) {
> try {
> this.connection.getMaster();
> break;
> } catch (MasterNotRunningException mnre) {
> HConnectionManager.deleteStaleConnection(this.connection);
> this.connection = HConnectionManager.getConnection(this.conf);
> } catch (UndeclaredThrowableException ute) {
> HConnectionManager.deleteStaleConnection(this.connection);
> this.connection = HConnectionManager.getConnection(this.conf);
> }
> try { // Sleep
> Thread.sleep(getPauseTime(tries));
> } catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> // we should delete connection between client and zookeeper
> HConnectionManager.deleteStaleConnection(this.connection);
> throw new MasterNotRunningException("Interrupted");
> }
> }
> if (tries >= numRetries) {
> // we should delete connection between client and zookeeper
> HConnectionManager.deleteStaleConnection(this.connection);
> throw new MasterNotRunningException("Retried " + numRetries + " times");
> }
> }
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165621#comment-13165621 ] stack commented on HBASE-4682: -- Thanks for explaination. +1 on commit. > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165606#comment-13165606 ] Lars Hofhansl commented on HBASE-4682: -- Heh... It takes 46 secs on my machine. Can make it large, too. As for the messy-ness... The problem is that Delete has strange rules as what you can do in a single Delete (when you add a deleteFamily all prior deleteColumn(s) are removed, etc). So deletes have to be written one-by-one to be correct (that is also why I added the Delete(kv) constructor, rather than an delete(kv) method). A put, on the other hand, allows adding all put-related KVs of the same row into a single Put object. That's why delete-kvs are issued immediately, and put-kvs are collected into a single Put (that's also what the existing code does). It is possible that particular row only has delete markers, so the Put is created when needed, and only written to the HBase when there were any put-kvs. > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Resolution: Fixed Status: Resolved (was: Patch Available) > index.html - found another grammatical error > > > Key: HBASE-4986 > URL: https://issues.apache.org/jira/browse/HBASE-4986 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: index_4986.xml.patch > > > "modeled after Google'" should be "modeled after Google's" - it was missing > the "s" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Attachment: index_4986.xml.patch > index.html - found another grammatical error > > > Key: HBASE-4986 > URL: https://issues.apache.org/jira/browse/HBASE-4986 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: index_4986.xml.patch > > > "modeled after Google'" should be "modeled after Google's" - it was missing > the "s" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4986) index.html - found another grammatical error
[ https://issues.apache.org/jira/browse/HBASE-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4986: - Status: Patch Available (was: Open) > index.html - found another grammatical error > > > Key: HBASE-4986 > URL: https://issues.apache.org/jira/browse/HBASE-4986 > Project: HBase > Issue Type: Bug >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Trivial > Attachments: index_4986.xml.patch > > > "modeled after Google'" should be "modeled after Google's" - it was missing > the "s" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4986) index.html - found another grammatical error
index.html - found another grammatical error Key: HBASE-4986 URL: https://issues.apache.org/jira/browse/HBASE-4986 Project: HBase Issue Type: Bug Reporter: Doug Meil Assignee: Doug Meil Priority: Trivial "modeled after Google'" should be "modeled after Google's" - it was missing the "s" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165590#comment-13165590 ] stack commented on HBASE-4981: -- +1 > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165591#comment-13165591 ] stack commented on HBASE-4981: -- Oh, do you need to mention new facility in the Scan help for shell? If you can get it in, add on commit. > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4981: - Attachment: 4981-v2.txt Patch including help for scan command. > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt, 4981-v2.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[
https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165589#comment-13165589
]
stack commented on HBASE-4682:
--
Nice test. For sure its medium sized? Takes < 50 seconds?
This bit seems a little messy:
{code}
-put.add(kv);
+if (kv.isDelete()) {
+ context.write(key, new Delete(kv));
+} else {
+ if (put == null) {
+put = new Put(key.get());
+ }
+ put.add(kv);
+}
}
- return put;
+ if (put != null) {
+context.write(key, put);
+ }
{code}
We write the context if its a delete but if a put, we make a put instance and
then do the context.write later... can they not both do context.write in same
place or share the context.write invocation (if context.write takes a Mutation).
> Support deleted rows using Import/Export
>
>
> Key: HBASE-4682
> URL: https://issues.apache.org/jira/browse/HBASE-4682
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0
>
> Attachments: 4682-v1.txt, 4682-v2.txt
>
>
> Parent allows keeping deleted rows around. Would be nice if those could be
> exported and imported as well.
> All the building blocks are there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Resolution: Fixed Status: Resolved (was: Patch Available) > pom.xml - changing docbook section depth to 2 for chapters > -- > > Key: HBASE-4985 > URL: https://issues.apache.org/jira/browse/HBASE-4985 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: pom_hbase_4985.xml.patch > > > pom.xml > * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, > but still only 1 level in the main TOC). > - 1 > + 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Attachment: pom_hbase_4985.xml.patch > pom.xml - changing docbook section depth to 2 for chapters > -- > > Key: HBASE-4985 > URL: https://issues.apache.org/jira/browse/HBASE-4985 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: pom_hbase_4985.xml.patch > > > pom.xml > * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, > but still only 1 level in the main TOC). > - 1 > + 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Status: Patch Available (was: Open) > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4985: - Status: Patch Available (was: Open) > pom.xml - changing docbook section depth to 2 for chapters > -- > > Key: HBASE-4985 > URL: https://issues.apache.org/jira/browse/HBASE-4985 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: pom_hbase_4985.xml.patch > > > pom.xml > * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, > but still only 1 level in the main TOC). > - 1 > + 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-4682: - Attachment: 4682-v2.txt Addressed Ted's comment. Added an Import/Export test (there were NO tests for Import/Export before) > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt, 4682-v2.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
[ https://issues.apache.org/jira/browse/HBASE-4985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165583#comment-13165583 ] stack commented on HBASE-4985: -- +1 > pom.xml - changing docbook section depth to 2 for chapters > -- > > Key: HBASE-4985 > URL: https://issues.apache.org/jira/browse/HBASE-4985 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: pom_hbase_4985.xml.patch > > > pom.xml > * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, > but still only 1 level in the main TOC). > - 1 > + 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4985) pom.xml - changing docbook section depth to 2 for chapters
pom.xml - changing docbook section depth to 2 for chapters -- Key: HBASE-4985 URL: https://issues.apache.org/jira/browse/HBASE-4985 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor pom.xml * changing to tocMaxDepth to 2 for chapters (will show 2 levels in Chapters, but still only 1 level in the main TOC). - 1 + 2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165565#comment-13165565 ] Lars Hofhansl commented on HBASE-4970: -- V2 looks good to me. +1 Is this good enough for you gaojinchao? It will put more burden on the application to manage the ExecutorService and HConnection. > Allow better control of resource consumption in HTable (backport HBASE-4805 > to 0.90 branch) > --- > > Key: HBASE-4970 > URL: https://issues.apache.org/jira/browse/HBASE-4970 > Project: HBase > Issue Type: Improvement > Components: client >Affects Versions: 0.90.4 >Reporter: gaojinchao >Assignee: gaojinchao >Priority: Trivial > Fix For: 0.90.5 > > Attachments: HBASE-4970_Branch90.patch, > HBASE-4970_Branch90_V1_trial.patch > > > In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES > is slowed down. > Why increasing keepAliveTime of HBase thread pool is slowing down our problem > occurance [RES value increase]? > You can go through the source of sun.nio.ch.Util. Every thread hold 3 > softreference of direct buffer(mustangsrc) for reusage. The code names the 3 > softreferences buffercache. If the buffer was all occupied or none was > suitable in size, and new request comes, new direct buffer is allocated. > After the service, the bigger one replaces the smaller one in buffercache. > The replaced buffer is released. > So I think we can add a parameter to change keepAliveTime of Htable thread > pool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4984) index.html - adding a link to architecture overview in "more information" section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165548#comment-13165548 ] Hudson commented on HBASE-4984: --- Integrated in HBase-TRUNK #2530 (See [https://builds.apache.org/job/HBase-TRUNK/2530/]) hbase-4984 index.html. adding link to arch overview in More Info section. > index.html - adding a link to architecture overview in "more information" > section. > -- > > Key: HBASE-4984 > URL: https://issues.apache.org/jira/browse/HBASE-4984 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: index_hbase_4984.xml.patch > > > index.html > * adding link to "architecture overview" in the book from index.html in the > More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[ https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165540#comment-13165540 ] Lars Hofhansl commented on HBASE-4981: -- Nice! The debug option would be extremely helpful. So I'll add the RAW stuff to the scan's help blurb, and then you'll continue in HBASE-4981? Do I get a +1 here? :) > add raw scan support to shell > - > > Key: HBASE-4981 > URL: https://issues.apache.org/jira/browse/HBASE-4981 > Project: HBase > Issue Type: Sub-task > Components: shell >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4981-v1.txt > > > Parent adds "raw" scan support to include delete markers and deleted rows in > scan results. Would be nice it that would available in the shell to see > exactly what exists in a table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4981) add raw scan support to shell
[
https://issues.apache.org/jira/browse/HBASE-4981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165526#comment-13165526
]
stack commented on HBASE-4981:
--
This is sweet. I like passing attribute at same time. Then could have RAW =>
true and DEBUG => true. Issue I was talking of was HBASE-4981, a debug filter.
I was thinking it would something like the following (assuming both RAW and
DEBUG):
{code}
hbase(main):005:0> scan 'x1', {RAW=>true, VERSIONS=>10}
ROW COLUMN+CELL
r2 column=f:, timestamp=1323323616226, type=DeleteFamily
r2 column=f:c, timestamp=1323323617759, value=v3
r2 column=f:c, timestamp=1323323616226, value=v2
r2 column=f:c, timestamp=1323323614496, value=v1
r2 column=f:.values_skipped_returning_this_row,
timestamp=1323323614496, value=20021
r2 column=f:.storefiles_in_this_family,
timestamp=1323323614496, value=4
...
2 row(s) in 0.0500 seconds
{code}
i.e. we'd tag on some values on end of each row result that described things
like number of storefiles in the cf, and how many values we skipped returning a
row result (for example, if lots of deleted values skipped, would be nice to
show that somehow)
> add raw scan support to shell
> -
>
> Key: HBASE-4981
> URL: https://issues.apache.org/jira/browse/HBASE-4981
> Project: HBase
> Issue Type: Sub-task
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.94.0
>
> Attachments: 4981-v1.txt
>
>
> Parent adds "raw" scan support to include delete markers and deleted rows in
> scan results. Would be nice it that would available in the shell to see
> exactly what exists in a table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165482#comment-13165482
]
Phabricator commented on HBASE-4969:
stack has commented on the revision "HBASE-4969 [jira] tautology in
HRegionInfo.readFields".
@Nicolas
That made a big difference (I'll doc this in our book after I get it to
work). After installing cert which the tool guided me through nicely, I got to
this:
Exception:
ERR-CONDUIT-CORE: Bogus ID provided to load().
(Run with --trace for a full exception trace.)
h-24-30:hbase stack$ arc amend --trace --revision D669
Loading phutil library 'arclib' from
'/Users/Stack/checkouts/hbase/.arc_jira_lib'...
>>> [0] conduit.connect()
<<< [0] 418,950 us
>>> [1] $ (cd '/Users/Stack/checkouts/hbase'; git rev-parse
--show-cdup)
<<< [1] 10,368 us
>>> [2] $ (cd '/Users/Stack/checkouts/hbase/'; git rev-parse --verify
HEAD^)
<<< [2] 15,268 us
>>> [3] $ (cd '/Users/Stack/checkouts/hbase/'; git diff --no-ext-diff
--no-textconv --raw 'HEAD^' --)
>>> [4] $ (cd '/Users/Stack/checkouts/hbase/'; git diff --no-ext-diff
--no-textconv --raw HEAD --)
>>> [5] $ (cd '/Users/Stack/checkouts/hbase/'; git ls-files --others
--exclude-standard)
>>> [6] $ (cd '/Users/Stack/checkouts/hbase/'; git ls-files -m)
<<< [6] 62,638 us
<<< [3] 67,122 us
<<< [5] 65,706 us
<<< [4] 75,585 us
>>> [7] differential.getcommitmessage()
Fatal error: Uncaught exception 'ConduitClientException' with message
'ERR-CONDUIT-CORE: Bogus ID provided to load().' in
/Users/Stack/checkouts/libphutil/src/conduit/client/ConduitFuture.php:54
Stack trace:
#0 /Users/Stack/checkouts/libphutil/src/future/proxy/FutureProxy.php(62):
ConduitFuture->didReceiveResult(Array)
#1 /Users/Stack/checkouts/libphutil/src/future/proxy/FutureProxy.php(39):
FutureProxy->getResult()
#2 /Users/Stack/checkouts/libphutil/src/conduit/client/ConduitClient.php(52):
FutureProxy->resolve()
#3
/Users/Stack/checkouts/arcanist/src/workflow/amend/ArcanistAmendWorkflow.php(125):
ConduitClient->callMethodSynchronous('differential.ge...', Array)
#4 /Users/Stack/checkouts/arcanist/scripts/arcanist.php(257):
ArcanistAmendWorkflow->run()
#5 {main}
thrown in
/Users/Stack/checkouts/libphutil/src/conduit/client/ConduitFuture.php on line 54
REVISION DETAIL
https://reviews.facebook.net/D669
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165468#comment-13165468
]
Phabricator commented on HBASE-4969:
nspiegelberg has commented on the revision "HBASE-4969 [jira] tautology in
HRegionInfo.readFields".
@stack: have you run 'mvn initialize -Darc' on that directory yet?
REVISION DETAIL
https://reviews.facebook.net/D669
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-4982. -- Resolution: Fixed Fix Version/s: 0.90.5 Assignee: stack Committed to TRUNK and the two branches. > graceful_stop.sh does not pass on the --config its passed to its internal > invocations of other hbase scripts > > > Key: HBASE-4982 > URL: https://issues.apache.org/jira/browse/HBASE-4982 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: stack >Assignee: stack > Fix For: 0.90.5 > > Attachments: 4982.txt > > > Means, unless conf is in default location, we mess up asking zk for state > (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4682) Support deleted rows using Import/Export
[ https://issues.apache.org/jira/browse/HBASE-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165458#comment-13165458 ] Lars Hofhansl commented on HBASE-4682: -- I did some more testing with this and everything works fine. Apparently there are no tests for import/export anywhere, so I am adding a test for existing functionality as well. > Support deleted rows using Import/Export > > > Key: HBASE-4682 > URL: https://issues.apache.org/jira/browse/HBASE-4682 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Affects Versions: 0.94.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.94.0 > > Attachments: 4682-v1.txt > > > Parent allows keeping deleted rows around. Would be nice if those could be > exported and imported as well. > All the building blocks are there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4982) graceful_stop.sh does not pass on the --config its passed to its internal invocations of other hbase scripts
[ https://issues.apache.org/jira/browse/HBASE-4982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4982: - Attachment: 4982.txt Tried this. It works. > graceful_stop.sh does not pass on the --config its passed to its internal > invocations of other hbase scripts > > > Key: HBASE-4982 > URL: https://issues.apache.org/jira/browse/HBASE-4982 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: stack > Attachments: 4982.txt > > > Means, unless conf is in default location, we mess up asking zk for state > (we'll be pointed at wrong ensemble), etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165451#comment-13165451
]
Phabricator commented on HBASE-4969:
stack has commented on the revision "HBASE-4969 [jira] tautology in
HRegionInfo.readFields".
How I fix this Prakash?
h-24-30:trunk stack$ arc amend --revision D669
Usage Exception: Failed to load library "arclib" at location ".arc_jira_lib".
Please check the "phutil_libraries" setting in your .arcconfig file. Refer to
page http://phabricator.com/docs/arcanist/article/Setting_Up_.arcconfig.html
for more info.
I'm trying to do what Mikhail asked me do,
REVISION DETAIL
https://reviews.facebook.net/D669
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165448#comment-13165448
]
Hadoop QA commented on HBASE-4980:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506636/0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 74 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/475//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/475//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/475//console
This message is automatically generated.
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
> Labels: newbie
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4655) Document architecture of backups
[ https://issues.apache.org/jira/browse/HBASE-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165436#comment-13165436 ] Karthik Ranganathan commented on HBASE-4655: @Doug: << list all the regions, for each region, ask the RS hosting it for a list of HFiles >> There is already an API to get a list of regions and the regionservers hosting them. And we added a new API to the RS to list the HFiles for the regions it hosts. << The strategy is great, but it will generate a flurry of (warranted) questions on how the average person does it. >> True - but this task is only to make sure the document is easy to read and understand by an average user. We can definitely add more details if needed, but that would risk confusing people. I will definitely incorporate the other suggestions (confusing names, etc). The rest of the tasks deal with giving a way for the average users to do backups by running/cron-ing a command and not have to deal with the internals of how it works. > Document architecture of backups > > > Key: HBASE-4655 > URL: https://issues.apache.org/jira/browse/HBASE-4655 > Project: HBase > Issue Type: Sub-task > Components: documentation, regionserver >Reporter: Karthik Ranganathan >Assignee: Karthik Ranganathan > Attachments: HBase Backups Architecture.docx > > > Basic idea behind the backup architecture for HBase -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165421#comment-13165421
]
Phabricator commented on HBASE-4969:
stack has accepted the revision "HBASE-4969 [jira] tautology in
HRegionInfo.readFields".
+1
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/HRegionInfo.java:740 Your fix looks
right Prakash.
REVISION DETAIL
https://reviews.facebook.net/D669
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165413#comment-13165413
]
stack commented on HBASE-4980:
--
If you see nothing in next hour, yes.
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
> Labels: newbie
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165406#comment-13165406
]
Shrijeet Paliwal commented on HBASE-4980:
-
Done attaching, should I click cancel patch and then click submit patch again?
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
> Labels: newbie
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4980) Null pointer exception in HBaseClient receiveResponse
[
https://issues.apache.org/jira/browse/HBASE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shrijeet Paliwal updated HBASE-4980:
Attachment: 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
> Null pointer exception in HBaseClient receiveResponse
> -
>
> Key: HBASE-4980
> URL: https://issues.apache.org/jira/browse/HBASE-4980
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.92.0
>Reporter: Shrijeet Paliwal
> Labels: newbie
> Attachments:
> 0001-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0002-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch,
> 0003-HBASE-4980-Fix-NPE-in-HBaseClient-receiveResponse.patch
>
>
> Relevant Stack trace:
> 2011-11-30 13:10:26,557 [IPC Client (47) connection to
> xx.xx.xx/172.22.4.68:60020 from an unknown user] WARN
> org.apache.hadoop.ipc.HBaseClient - Unexpected exception receiving call
> responses
> java.lang.NullPointerException
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:583)
> >-at
> >org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:511)
> {code}
> if (LOG.isDebugEnabled())
> LOG.debug(getName() + " got value #" + id);
> Call call = calls.remove(id);
> // Read the flag byte
> byte flag = in.readByte();
> boolean isError = ResponseFlag.isError(flag);
> if (ResponseFlag.isLength(flag)) {
> // Currently length if present is unused.
> in.readInt();
> }
> int state = in.readInt(); // Read the state. Currently unused.
> if (isError) {
> //noinspection ThrowableInstanceNeverThrown
> call.setException(new RemoteException( WritableUtils.readString(in),
> WritableUtils.readString(in)));
> } else {
> {code}
> This line {code}Call call = calls.remove(id);{code} may return a null
> 'call'. It is so because if you have rpc timeout enable, we proactively clean
> up other calls which have expired their lifetime along with the call for
> which socket timeout exception happend.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4983) index.html - correcting BigTable link, changing intro line, adding one feature
[ https://issues.apache.org/jira/browse/HBASE-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165403#comment-13165403 ] Hudson commented on HBASE-4983: --- Integrated in HBase-TRUNK #2529 (See [https://builds.apache.org/job/HBase-TRUNK/2529/]) hbase-4983. index.html: one correction, one change, one addition. > index.html - correcting BigTable link, changing intro line, adding one feature > -- > > Key: HBASE-4983 > URL: https://issues.apache.org/jira/browse/HBASE-4983 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: index_HBASE_4983.xml.patch > > > index.html > * correcting BigTable link (existing one was actually a 404!) > * Changed top-level description to "Think of it as a distributed scalable Big > Data store." > * Added a feature "linear and modular scalability" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4969) tautology in HRegionInfo.readFields
[
https://issues.apache.org/jira/browse/HBASE-4969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-4969:
---
Attachment: HBASE-4969.D669.1.patch
khemani requested code review of "HBASE-4969 [jira] tautology in
HRegionInfo.readFields".
Reviewers: nspiegelberg, dhruba, JIRA
as in title
TEST PLAN
minor change.
REVISION DETAIL
https://reviews.facebook.net/D669
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/HRegionInfo.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1479/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> tautology in HRegionInfo.readFields
> ---
>
> Key: HBASE-4969
> URL: https://issues.apache.org/jira/browse/HBASE-4969
> Project: HBase
> Issue Type: Bug
>Reporter: Prakash Khemani
>Assignee: Prakash Khemani
> Attachments: HBASE-4969.D669.1.patch
>
>
> In HRegionInfo.readFields() the following looks wrong to me
> } else if (getVersion() == VERSION) {
> it is always true.
> Should it have been
> } else if (getVersion() == version) {
> version is a local variable where the deserialized-version is stored.
> (I am struggling with another issue where after applying some patches -
> including "HBASE-4388 Second start after migration from 90 to trunk crashes"
> my version of hbase-92 HRegionInfo.readFields() tries to find HTD in
> HRegionInfo and fails)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4984) index.html - adding a link to architecture overview in "more information" section.
[ https://issues.apache.org/jira/browse/HBASE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Meil updated HBASE-4984: - Resolution: Fixed Status: Resolved (was: Patch Available) > index.html - adding a link to architecture overview in "more information" > section. > -- > > Key: HBASE-4984 > URL: https://issues.apache.org/jira/browse/HBASE-4984 > Project: HBase > Issue Type: Improvement >Reporter: Doug Meil >Assignee: Doug Meil >Priority: Minor > Attachments: index_hbase_4984.xml.patch > > > index.html > * adding link to "architecture overview" in the book from index.html in the > More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4984) index.html - adding a link to architecture overview in "more information" section.
index.html - adding a link to architecture overview in "more information" section. -- Key: HBASE-4984 URL: https://issues.apache.org/jira/browse/HBASE-4984 Project: HBase Issue Type: Improvement Reporter: Doug Meil Assignee: Doug Meil Priority: Minor Attachments: index_hbase_4984.xml.patch index.html * adding link to "architecture overview" in the book from index.html in the More Information? section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
