[jira] [Commented] (HBASE-8480) Embed HDFS into HBase
[ https://issues.apache.org/jira/browse/HBASE-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648231#comment-13648231 ] Lars George commented on HBASE-8480: [~otis] I agree that this is something that needs to be carefully thought about. But on the other hand a lot of things have happened since then, HBase and HDFS have seen many optimizations that makes this more feasible today. Sure, I agree with what [~streamy] commented in the thread, we will see users overdo it caused by additional overload of the same single or small cluster setup. But that was always the case. I believe we need to also compete with the smaller, easier to get going with NoSQL solutions. But then we also should have a Bootstrap backed website :) > Embed HDFS into HBase > - > > Key: HBASE-8480 > URL: https://issues.apache.org/jira/browse/HBASE-8480 > Project: HBase > Issue Type: New Feature >Reporter: Lars George > > HBase is often a bit more involved to get going. We already have the option > to host ZooKeeper for very small clusters. We should have the same for HDFS. > The idea is that it adjusts replication based on the number of nodes, i.e. > from 1 to 3 (the default), so that you could start with a single node and > grow the cluster from there. Once the cluster reaches a certain size, and the > admin decides to split the components, we should have a why to export the > proper configs/settings so that you can easily start up an external HDFS > and/or ZooKeeper, while updating the HBase config as well to point to the new > "locations". > The goal is to start a fully operational HBase that can grow from single > machine to multi machine clusters with just a single daemon on each machine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.95. Thanks for review Sergey and saving me from myself. > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt, 8430v3.txt, 8430v4trunk.txt, > 8430v4.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Attachment: 8430v4trunk.txt What I committed to trunk. > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt, 8430v3.txt, 8430v4trunk.txt, > 8430v4.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Attachment: 8430v4.txt What I ended up committing to 0.95; small differences around imports. > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt, 8430v3.txt, 8430v4.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8480) Embed HDFS into HBase
[ https://issues.apache.org/jira/browse/HBASE-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648220#comment-13648220 ] Lars George commented on HBASE-8480: [~stack] Yes, that makes sense, we need to define the various steps. I mean, I think for development it is good to have the local and pseudo modes as before. We would just add another choice, i.e. spinning up HDFS as well inside the nodes. Now, going from one to two and then three servers is the tricky part. By default, if you unpack a tarball and spin up HBase, I would stay in local mode as usual. Same for the pseudo distributed mode, i.e. the user will need to configure this as needed herself. But then if wanted we could have a little CLI config wizard, that preps the new fully autonomous mode of HBase, where everything is controlled by it, but has all the cluster components. Once you start this, you would either have one server that runs a Master+NameNode process plus a RegionServer+DataNode process. Or if we had the above Master-less option, we can run one single process with all inside - these are just semantics I'd say. Though I personally would love to see the latter eventually to make HBase a single process system to boot with. Then as you add nodes, you will have to simply join a new RS+DN process on another node and the config should be amended - but also could stay the same and the dfs.replication factor set to "automatic", which scales it from 1 to "default" (or a maximum we can specify). In other words, when you run two nodes, then you are setting the replication to 2, with three nodes to 3, with four nodes leave it at 3 and so on. That should also include a flag that says when ramping up or down the replication factor that it should apply this to all the files in HDFS. That way you can increase and decrease the nodes as needed. As for splitting out the HBM+NN to a separate machine, that is really meaning to shut down the RS+DN process or internal thread on that machine. Adding a SNN or HA NN is then a little beyond the automated scope. Well, the SNN we will need, so that should be handled since clusters are meant to be up forever/a long time. But reconfiguring to non-automatic mode is then done using the CLI wizard or editing the configs, followed by a rolling restart. bq. I like the idea of bundling the master and regionserver in one binary better; no more special master treatment... any one can be msster and or a regionserver? I do believe that is only useful in automatic mode, because on a larger cluster with dedicated roles, you already have 2-3 master machines running NNs, ZKs and so on. Then adding Master's there is trivial (since it is all automatically deployed most of the time by admins). That, methinks, would not really have a tangible advantage. But yes, when things are small, this is certainly something that we should have. See above. > Embed HDFS into HBase > - > > Key: HBASE-8480 > URL: https://issues.apache.org/jira/browse/HBASE-8480 > Project: HBase > Issue Type: New Feature >Reporter: Lars George > > HBase is often a bit more involved to get going. We already have the option > to host ZooKeeper for very small clusters. We should have the same for HDFS. > The idea is that it adjusts replication based on the number of nodes, i.e. > from 1 to 3 (the default), so that you could start with a single node and > grow the cluster from there. Once the cluster reaches a certain size, and the > admin decides to split the components, we should have a why to export the > proper configs/settings so that you can easily start up an external HDFS > and/or ZooKeeper, while updating the HBase config as well to point to the new > "locations". > The goal is to start a fully operational HBase that can grow from single > machine to multi machine clusters with just a single daemon on each machine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8488) HBase transitive dependencies not being pulled in when building apps like Flume which depend on HBase
[
https://issues.apache.org/jira/browse/HBASE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648215#comment-13648215
]
Enis Soztutar commented on HBASE-8488:
--
I think the problem kicks in because when you refer the installed pom as a
dependency, then no profile kicks in, and hence compat.module and the runtime
dependency cannot be resolved. We did some testing with roshan to add
hadoop1|2-compat modules as normal modules (not as defined inside profiles),
but did not do change the hbase-server's dependency on compat as a runtime
dependency.
> HBase transitive dependencies not being pulled in when building apps like
> Flume which depend on HBase
> -
>
> Key: HBASE-8488
> URL: https://issues.apache.org/jira/browse/HBASE-8488
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.95.0
>Reporter: Roshan Naik
>
> Here is a snippet of the errors seen when building against Hbase
> {code}
> [WARNING] Invalid POM for org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details: Some problems were encountered while processing the POMs:
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not
> match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 982, column 21
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does
> not match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 987, column 21
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648207#comment-13648207
]
Lars Hofhansl commented on HBASE-7122:
--
Looks good. Will commit to 0.94 tomorrow if I hear no objections.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: HBase-7122-94.patch, HBase-7122-94-v2.patch,
> HBase-7122-95.patch, HBase-7122-95-v2.patch, HBase-7122-95-v3.patch,
> HBase-7122-95-v4.patch, HBase-7122.patch, HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8488) HBase transitive dependencies not being pulled in when building apps like Flume which depend on HBase
[
https://issues.apache.org/jira/browse/HBASE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648184#comment-13648184
]
stack commented on HBASE-8488:
--
If you define it on the command line when building, will that work?
How are you refering to hbase in your pom? Are you trying to build against the
SNAPSHOTS for hadoop1 and hadoop2?
Thanks.
> HBase transitive dependencies not being pulled in when building apps like
> Flume which depend on HBase
> -
>
> Key: HBASE-8488
> URL: https://issues.apache.org/jira/browse/HBASE-8488
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.95.0
>Reporter: Roshan Naik
>
> Here is a snippet of the errors seen when building against Hbase
> {code}
> [WARNING] Invalid POM for org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details: Some problems were encountered while processing the POMs:
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not
> match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 982, column 21
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does
> not match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 987, column 21
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8480) Embed HDFS into HBase
[ https://issues.apache.org/jira/browse/HBASE-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648173#comment-13648173 ] stack commented on HBASE-8480: -- /me I love when fellows quote the mailing list and the mailing list was good [~larsgeorge] So, we have minihbase which has everything in the one jvm. And then we have pseudo-distributed w/ all daemons on the one node... then we have full-blown cluster. How you want to bridge between these stages? How you think it would work? We'd always write dfs, never local fs? To go from standalone hbase, we'd allow a dn from another host join the cluster? How would we pass off Namenode responsibilities to a real namenode? I like the idea of bundling the master and regionserver in one binary better; no more special master treatment... any one can be msster and or a regionserver? That seems easier and would do a bunch of simplification? Would even help your project here? Thanks boss. > Embed HDFS into HBase > - > > Key: HBASE-8480 > URL: https://issues.apache.org/jira/browse/HBASE-8480 > Project: HBase > Issue Type: New Feature >Reporter: Lars George > > HBase is often a bit more involved to get going. We already have the option > to host ZooKeeper for very small clusters. We should have the same for HDFS. > The idea is that it adjusts replication based on the number of nodes, i.e. > from 1 to 3 (the default), so that you could start with a single node and > grow the cluster from there. Once the cluster reaches a certain size, and the > admin decides to split the components, we should have a why to export the > proper configs/settings so that you can easily start up an external HDFS > and/or ZooKeeper, while updating the HBase config as well to point to the new > "locations". > The goal is to start a fully operational HBase that can grow from single > machine to multi machine clusters with just a single daemon on each machine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8214: - Resolution: Fixed Fix Version/s: 0.95.1 Release Note: Take on the protobuf Service model. Undoes two layers of rpc -- a proxy that did reflection to hook up methods and an 'engine' layer that was put in place to support secure/insecure and Writables vs Protobuf rpc'ing. Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.95. Thanks for the security redo Gary. Thanks for reviews. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 0.95.1 > > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, > 8214v4.txt, 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, > 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[
https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648140#comment-13648140
]
Hadoop QA commented on HBASE-5746:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581647/HBASE-5746-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 12 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5547//console
This message is automatically generated.
> HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no
> checksums (0.96)
> -
>
> Key: HBASE-5746
> URL: https://issues.apache.org/jira/browse/HBASE-5746
> Project: HBase
> Issue Type: Sub-task
> Components: io, regionserver
>Reporter: Lars Hofhansl
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch,
> HBASE-5746-v1.patch, HBASE-5746-v2.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8488) HBase transitive dependencies not being pulled in when building apps like Flume which depend on HBase
[
https://issues.apache.org/jira/browse/HBASE-8488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648139#comment-13648139
]
Roshan Naik commented on HBASE-8488:
The problem initially appears to be indicate that ${compat.module} property is
not defined .. although it actually is (inside the hadoop 1/2 profiles in
parent pom). But these are unfortunately not visible when building apps that
depend on hbase.
> HBase transitive dependencies not being pulled in when building apps like
> Flume which depend on HBase
> -
>
> Key: HBASE-8488
> URL: https://issues.apache.org/jira/browse/HBASE-8488
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 0.95.0
>Reporter: Roshan Naik
>
> Here is a snippet of the errors seen when building against Hbase
> {code}
> [WARNING] Invalid POM for org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT,
> transitive dependencies (if any) will not be available, enable debug logging
> for more details: Some problems were encountered while processing the POMs:
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not
> match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 982, column 21
> [ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
> org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does
> not match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
> /Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
> line 987, column 21
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7244) Provide a command or argument to startup, that formats znodes if provided
[
https://issues.apache.org/jira/browse/HBASE-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-7244:
--
Status: Patch Available (was: Open)
> Provide a command or argument to startup, that formats znodes if provided
> -
>
> Key: HBASE-7244
> URL: https://issues.apache.org/jira/browse/HBASE-7244
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.94.0
>Reporter: Harsh J
>Assignee: rajeshbabu
>Priority: Critical
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: HBASE-7244_2.patch, HBASE-7244_3.patch,
> HBASE-7244_4.patch, HBASE-7244_5.patch, HBASE-7244_6.patch,
> HBASE-7244_7.patch, HBASE-7244.patch
>
>
> Many a times I've had to, and have seen instructions being thrown, to stop
> cluster, clear out ZK and restart.
> While this is only a quick (and painful to master) fix, it is certainly nifty
> to some smaller cluster users but the process is far too long, roughly:
> 1. Stop HBase
> 2. Start zkCli.sh and connect to the right quorum
> 3. Find and ensure the HBase parent znode from the configs (/hbase only by
> default)
> 4. Run an "rmr /hbase" in the zkCli.sh shell, or manually delete each znode
> if on a lower version of ZK.
> 5. Quit zkCli.sh and start HBase again
> Perhaps it may be useful, if the start-hbase.sh itself accepted a formatZK
> parameter. Such that, when you do a {{start-hbase.sh -formatZK}}, it does
> steps 2-4 automatically for you.
> For safety, we could make the formatter code ensure that no HBase instance is
> actually active, and skip the format process if it is. Similar to a HDFS
> NameNode's format, which would disallow if the name directories are locked.
> Would this be a useful addition for administrators? Bigtop too can provide a
> service subcommand that could do this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7244) Provide a command or argument to startup, that formats znodes if provided
[
https://issues.apache.org/jira/browse/HBASE-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-7244:
--
Status: Open (was: Patch Available)
> Provide a command or argument to startup, that formats znodes if provided
> -
>
> Key: HBASE-7244
> URL: https://issues.apache.org/jira/browse/HBASE-7244
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.94.0
>Reporter: Harsh J
>Assignee: rajeshbabu
>Priority: Critical
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: HBASE-7244_2.patch, HBASE-7244_3.patch,
> HBASE-7244_4.patch, HBASE-7244_5.patch, HBASE-7244_6.patch,
> HBASE-7244_7.patch, HBASE-7244.patch
>
>
> Many a times I've had to, and have seen instructions being thrown, to stop
> cluster, clear out ZK and restart.
> While this is only a quick (and painful to master) fix, it is certainly nifty
> to some smaller cluster users but the process is far too long, roughly:
> 1. Stop HBase
> 2. Start zkCli.sh and connect to the right quorum
> 3. Find and ensure the HBase parent znode from the configs (/hbase only by
> default)
> 4. Run an "rmr /hbase" in the zkCli.sh shell, or manually delete each znode
> if on a lower version of ZK.
> 5. Quit zkCli.sh and start HBase again
> Perhaps it may be useful, if the start-hbase.sh itself accepted a formatZK
> parameter. Such that, when you do a {{start-hbase.sh -formatZK}}, it does
> steps 2-4 automatically for you.
> For safety, we could make the formatter code ensure that no HBase instance is
> actually active, and skip the format process if it is. Similar to a HDFS
> NameNode's format, which would disallow if the name directories are locked.
> Would this be a useful addition for administrators? Bigtop too can provide a
> service subcommand that could do this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7244) Provide a command or argument to startup, that formats znodes if provided
[
https://issues.apache.org/jira/browse/HBASE-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-7244:
--
Attachment: HBASE-7244_7.patch
[~stack]
bq. The default, if you pass no arguments, is to clean everything?
No it will not do any thing. Any way added usage in this patch.
bq.Make it so you have to pass in what you want cleaned? Pass --cleanZk to
clean zk, – cleanHDFS to clean hdfs, and --cleanAll to clean all?
changed.
Thanks.
> Provide a command or argument to startup, that formats znodes if provided
> -
>
> Key: HBASE-7244
> URL: https://issues.apache.org/jira/browse/HBASE-7244
> Project: HBase
> Issue Type: New Feature
> Components: Zookeeper
>Affects Versions: 0.94.0
>Reporter: Harsh J
>Assignee: rajeshbabu
>Priority: Critical
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: HBASE-7244_2.patch, HBASE-7244_3.patch,
> HBASE-7244_4.patch, HBASE-7244_5.patch, HBASE-7244_6.patch,
> HBASE-7244_7.patch, HBASE-7244.patch
>
>
> Many a times I've had to, and have seen instructions being thrown, to stop
> cluster, clear out ZK and restart.
> While this is only a quick (and painful to master) fix, it is certainly nifty
> to some smaller cluster users but the process is far too long, roughly:
> 1. Stop HBase
> 2. Start zkCli.sh and connect to the right quorum
> 3. Find and ensure the HBase parent znode from the configs (/hbase only by
> default)
> 4. Run an "rmr /hbase" in the zkCli.sh shell, or manually delete each znode
> if on a lower version of ZK.
> 5. Quit zkCli.sh and start HBase again
> Perhaps it may be useful, if the start-hbase.sh itself accepted a formatZK
> parameter. Such that, when you do a {{start-hbase.sh -formatZK}}, it does
> steps 2-4 automatically for you.
> For safety, we could make the formatter code ensure that no HBase instance is
> actually active, and skip the format process if it is. Similar to a HDFS
> NameNode's format, which would disallow if the name directories are locked.
> Would this be a useful addition for administrators? Bigtop too can provide a
> service subcommand that could do this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8488) HBase transitive dependencies not being pulled in when building apps like Flume which depend on HBase
Roshan Naik created HBASE-8488:
--
Summary: HBase transitive dependencies not being pulled in when
building apps like Flume which depend on HBase
Key: HBASE-8488
URL: https://issues.apache.org/jira/browse/HBASE-8488
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.95.0
Reporter: Roshan Naik
Here is a snippet of the errors seen when building against Hbase
{code}
[WARNING] Invalid POM for org.apache.hbase:hbase-common:jar:0.97.0-SNAPSHOT,
transitive dependencies (if any) will not be available, enable debug logging
for more details: Some problems were encountered while processing the POMs:
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not
match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
/Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
line 982, column 21
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does
not match a valid id pattern. @ org.apache.hbase:hbase:0.97.0-SNAPSHOT,
/Users/rnaik/.m2/repository/org/apache/hbase/hbase/0.97.0-SNAPSHOT/hbase-0.97.0-SNAPSHOT.pom,
line 987, column 21
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648117#comment-13648117
]
Enis Soztutar commented on HBASE-8314:
--
Thanks Jimmy, I failed to see that in the patch.
bq. However, according to HBASE-8389, HBASE-7878 may not always work.
That is not my takeaway from N's comments here:
https://issues.apache.org/jira/browse/HBASE-8389?focusedCommentId=13642669&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13642669
> HLogSplitter can retry to open a 0-length hlog file
> ---
>
> Key: HBASE-8314
> URL: https://issues.apache.org/jira/browse/HBASE-8314
> Project: HBase
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.98.0, 0.95.1
>
> Attachments: region-server.log, trunk-8314.patch, trunk-8314_v2.patch
>
>
> In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
> fail to open it since it can get the file length, therefore, master can't
> start.
> {noformat}
> java.io.IOException: Cannot obtain block length for LocatedBlock{...;
> getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:117)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648112#comment-13648112 ] Enis Soztutar commented on HBASE-5746: -- Thanks for the test. +1 on commit except for Ted's comment. Referring arbitrary jira numbers makes it harder to reason about context some time later. > HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no > checksums (0.96) > - > > Key: HBASE-5746 > URL: https://issues.apache.org/jira/browse/HBASE-5746 > Project: HBase > Issue Type: Sub-task > Components: io, regionserver >Reporter: Lars Hofhansl >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 0.95.1 > > Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch, > HBASE-5746-v1.patch, HBASE-5746-v2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Status: Patch Available (was: Open) > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > Attachments: HBASE-8487-v1.patch > > Original Estimate: 20m > Remaining Estimate: 20m > > Wrong description about regionservers in "2.4. Example configurations". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8253) A corrupted log blocked ReplicationSource
[
https://issues.apache.org/jira/browse/HBASE-8253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648093#comment-13648093
]
Jieshan Bean commented on HBASE-8253:
-
bq.how come you got this error in a normal source?
Yes, it happened in a normal source not a recovered one. Primary data node was
forcely powered down, so logRoll was requested. And during that time, only
partial of last edit was written. Then we saw this problem.
> A corrupted log blocked ReplicationSource
> -
>
> Key: HBASE-8253
> URL: https://issues.apache.org/jira/browse/HBASE-8253
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.6
>Reporter: Jieshan Bean
>Assignee: Jieshan Bean
> Attachments: HBASE-8253-94.patch
>
>
> A writting log got corrupted when we forcely power down one node. Only
> partial of last WALEdit was written into that log. And that log was not the
> last one in replication queue.
> ReplicationSource was blocked under this scenario. A lot of logs like below
> were printed:
> {noformat}
> 2013-03-30 06:53:48,628 WARN
> [regionserver26003-EventThread.replicationSource,1] 1 Got:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:334)
> java.io.EOFException:
> hdfs://hacluster/hbase/.logs/master11,26003,1364530862620/master11%2C26003%2C1364530862620.1364553936510,
> entryStart=40434738, pos=40450048, end=40450048, edit=0
> at sun.reflect.GeneratedConstructorAccessor42.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.addFileInfoToException(SequenceFileLogReader.java:295)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:240)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.readNextAndSetPosition(ReplicationHLogReaderManager.java:84)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:412)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:330)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:68)
> at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:106)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2282)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2181)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2227)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:238)
> ... 3 more
> ..
> 2013-03-30 06:54:38,899 WARN
> [regionserver26003-EventThread.replicationSource,1] 1 Got:
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:334)
> java.io.EOFException:
> hdfs://hacluster/hbase/.logs/master11,26003,1364530862620/master11%2C26003%2C1364530862620.1364553936510,
> entryStart=40434738, pos=40450048, end=40450048, edit=0
> at sun.reflect.GeneratedConstructorAccessor42.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.addFileInfoToException(SequenceFileLogReader.java:295)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.next(SequenceFileLogReader.java:240)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.readNextAndSetPosition(ReplicationHLogReaderManager.java:84)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:412)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:330)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:68)
> at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:106)
> at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:2282)
> at org.apache.hadoop.io.SequenceFile$Reader.ne
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Attachment: HBASE-8487-v1.patch > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > Attachments: HBASE-8487-v1.patch > > Original Estimate: 20m > Remaining Estimate: 20m > > Wrong description about regionservers in "2.4. Example configurations". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Remaining Estimate: 20m Original Estimate: 20m > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > Original Estimate: 20m > Remaining Estimate: 20m > > Wrong description about regionservers in "2.4. Example configurations". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Description: Wrong description about regionservers in "2.4. Example configurations". (was: Wrong description about regionservers in 2.4. Example configurations) > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > > Wrong description about regionservers in "2.4. Example configurations". -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Description: Wrong description about regionservers in 2.4. Example configurations (was: Wrong description of regionservers file in Section "2.4. Example configurations") > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > > Wrong description about regionservers in 2.4. Example configurations -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8487) Wrong description about regionservers in 2.4. Example configurations
[ https://issues.apache.org/jira/browse/HBASE-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingguo Yao updated HBASE-8487: --- Summary: Wrong description about regionservers in 2.4. Example configurations (was: Wrong ) > Wrong description about regionservers in 2.4. Example configurations > > > Key: HBASE-8487 > URL: https://issues.apache.org/jira/browse/HBASE-8487 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 0.95.0 >Reporter: Jingguo Yao >Priority: Minor > > Wrong description of regionservers file in Section "2.4. Example > configurations" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8487) Wrong
Jingguo Yao created HBASE-8487: -- Summary: Wrong Key: HBASE-8487 URL: https://issues.apache.org/jira/browse/HBASE-8487 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.95.0 Reporter: Jingguo Yao Priority: Minor Wrong description of regionservers file in Section "2.4. Example configurations" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[
https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648082#comment-13648082
]
Ted Yu commented on HBASE-5746:
---
The new test passed.
{code}
+ public void test5746() throws Exception {
{code}
How about naming the new test testHeaderSizeOfHFileBlockWithoutChecksum ? You
can mention HBASE-5746 in comment.
> HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no
> checksums (0.96)
> -
>
> Key: HBASE-5746
> URL: https://issues.apache.org/jira/browse/HBASE-5746
> Project: HBase
> Issue Type: Sub-task
> Components: io, regionserver
>Reporter: Lars Hofhansl
>Assignee: Sergey Shelukhin
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch,
> HBASE-5746-v1.patch, HBASE-5746-v2.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8446) Allow parallel snapshot of different tables
[ https://issues.apache.org/jira/browse/HBASE-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648080#comment-13648080 ] Sergey Shelukhin commented on HBASE-8446: - +1.. could instead put some sort of placeholder there and remove when failing, but probably doesn't matter, there are no particularly slow ops in the middle > Allow parallel snapshot of different tables > --- > > Key: HBASE-8446 > URL: https://issues.apache.org/jira/browse/HBASE-8446 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.95.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 0.95.2 > > Attachments: HBASE-8446-94.patch, HBASE-8446-v0.patch, > HBASE-8446-v1.patch, HBASE-8446-v2.patch, HBASE-8446-v3.patch, > HBASE-8446-v4.patch, HBASE-8446-v5.patch, HBASE-8446-v6.patch > > > currently only one snapshot at the time is allowed. > Like for the restore, we should allow taking snapshot of different tables in > parallel. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648077#comment-13648077 ] Sergey Shelukhin commented on HBASE-8430: - looks reasonable > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt, 8430v3.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-5746: Attachment: HBASE-5746-v2.patch added test > HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no > checksums (0.96) > - > > Key: HBASE-5746 > URL: https://issues.apache.org/jira/browse/HBASE-5746 > Project: HBase > Issue Type: Sub-task > Components: io, regionserver >Reporter: Lars Hofhansl >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 0.95.1 > > Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch, > HBASE-5746-v1.patch, HBASE-5746-v2.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8465) Auto-drop rollback snapshot for snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648072#comment-13648072
]
Ted Yu commented on HBASE-8465:
---
Looking through
http://dev.mysql.com/doc/refman/5.1/en/point-in-time-recovery.html , I don't
see mentioning of failsafe snapshot (or its equivalent).
Will see how Oracle handles snapshot restore.
> Auto-drop rollback snapshot for snapshot restore
>
>
> Key: HBASE-8465
> URL: https://issues.apache.org/jira/browse/HBASE-8465
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8465-trunk-v1.txt, 8465-trunk-v2.txt
>
>
> Below is an excerpt from snapshot restore javadoc:
> {code}
>* Restore the specified snapshot on the original table. (The table must be
> disabled)
>* Before restoring the table, a new snapshot with the current table state
> is created.
>* In case of failure, the table will be rolled back to the its original
> state.
> {code}
> We can improve the handling of rollbackSnapshot in two ways:
> 1. give better name to the rollbackSnapshot (adding
> {code}'-for-rollback-'{code}). Currently the name is of the form:
> String rollbackSnapshot = snapshotName + "-" +
> EnvironmentEdgeManager.currentTimeMillis();
> 2. drop rollbackSnapshot at the end of restoreSnapshot() if the restore is
> successful. We can introduce new config param, named
> 'hbase.snapshot.restore.drop.rollback', to keep compatibility with current
> behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[
https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648065#comment-13648065
]
Hadoop QA commented on HBASE-8430:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581639/8430v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5546//console
This message is automatically generated.
> Cell decoder/scanner/etc. should not hide exceptions
>
>
> Key: HBASE-8430
> URL: https://issues.apache.org/jira/browse/HBASE-8430
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, Protobufs
>Affects Versions: 0.95.0
>Reporter: Sergey Shelukhin
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8430.txt, 8430v2.txt, 8430v3.txt
>
>
> Cell scanner, base decoder, etc., hide IOException inside runtime exception.
> This can lead to unexpected behavior because a lot of code only expects
> IOException. There's no logical justification behind this hiding so it should
> be removed before it's too late (the sooner we do it the less throws
> declarations need to be added)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648062#comment-13648062
]
Lars Hofhansl commented on HBASE-8482:
--
We've mostly given up on replacing every call to System.currentTimeMillis() to
EEM.currentTimeMillis(). There were too many issues, and nobody had stepped up
again.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648058#comment-13648058 ] Enis Soztutar commented on HBASE-5746: -- If this is tested, how come we have not run into it. I though we should test it with having a small file w/o checksums and reading it into block cache with some encoding, no? > HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no > checksums (0.96) > - > > Key: HBASE-5746 > URL: https://issues.apache.org/jira/browse/HBASE-5746 > Project: HBase > Issue Type: Sub-task > Components: io, regionserver >Reporter: Lars Hofhansl >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 0.95.1 > > Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch, > HBASE-5746-v1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5746) HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no checksums (0.96)
[ https://issues.apache.org/jira/browse/HBASE-5746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648054#comment-13648054 ] Sergey Shelukhin commented on HBASE-5746: - This is already tested in backward compat test... which I have found out after trying to make changes to writer to write file w/o checksum unfortunately 0_o This patch is not about reading old files as such, it's about reading blocks to cache > HFileDataBlockEncoderImpl uses wrong header size when reading HFiles with no > checksums (0.96) > - > > Key: HBASE-5746 > URL: https://issues.apache.org/jira/browse/HBASE-5746 > Project: HBase > Issue Type: Sub-task > Components: io, regionserver >Reporter: Lars Hofhansl >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 0.95.1 > > Attachments: 5720-trunk-v2.txt, HBASE-5746-v0.patch, > HBASE-5746-v1.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[
https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648051#comment-13648051
]
Hadoop QA commented on HBASE-8214:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581630/8214v15.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 123
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift.TestThriftServer
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5545//console
This message is automatically generated.
> Remove proxy and engine, rely directly on pb generated Service
> --
>
> Key: HBASE-8214
> URL: https://issues.apache.org/jira/browse/HBASE-8214
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt,
> 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt,
> 8214v4.txt, 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch,
> 8214v8.txt, 8214v9.txt
>
>
> Attached patch is not done. Removes two to three layers -- depending on how
> you count -- between client and rpc client and similar on server side
> (between rpc and server implementation). Strips ProtobufRpcServer/Client and
> HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8480) Embed HDFS into HBase
[ https://issues.apache.org/jira/browse/HBASE-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648045#comment-13648045 ] Otis Gospodnetic commented on HBASE-8480: - +1 This is getting close to "HBase in a box" - http://search-hadoop.com/m/p68C12nb7Hn (2010): "HBase in a box" is like "dynamic equilibrium", or "virtual reality", or "jumbo shrimp"... :-) > Embed HDFS into HBase > - > > Key: HBASE-8480 > URL: https://issues.apache.org/jira/browse/HBASE-8480 > Project: HBase > Issue Type: New Feature >Reporter: Lars George > > HBase is often a bit more involved to get going. We already have the option > to host ZooKeeper for very small clusters. We should have the same for HDFS. > The idea is that it adjusts replication based on the number of nodes, i.e. > from 1 to 3 (the default), so that you could start with a single node and > grow the cluster from there. Once the cluster reaches a certain size, and the > admin decides to split the components, we should have a why to export the > proper configs/settings so that you can easily start up an external HDFS > and/or ZooKeeper, while updating the HBase config as well to point to the new > "locations". > The goal is to start a fully operational HBase that can grow from single > machine to multi machine clusters with just a single daemon on each machine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648042#comment-13648042
]
stack commented on HBASE-8478:
--
+1
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch, hbase-8478_v2.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8478:
-
Attachment: hbase-8478_v2.patch
How about this?
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch, hbase-8478_v2.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Attachment: 8430v3.txt Retry > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt, 8430v3.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[
https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648030#comment-13648030
]
Hadoop QA commented on HBASE-8430:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581619/8430v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 16 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5544//console
This message is automatically generated.
> Cell decoder/scanner/etc. should not hide exceptions
>
>
> Key: HBASE-8430
> URL: https://issues.apache.org/jira/browse/HBASE-8430
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC, Protobufs
>Affects Versions: 0.95.0
>Reporter: Sergey Shelukhin
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8430.txt, 8430v2.txt
>
>
> Cell scanner, base decoder, etc., hide IOException inside runtime exception.
> This can lead to unexpected behavior because a lot of code only expects
> IOException. There's no logical justification behind this hiding so it should
> be removed before it's too late (the sooner we do it the less throws
> declarations need to be added)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8483) HConnectionManager can leak ZooKeeper connections when using deleteStaleConnection
[ https://issues.apache.org/jira/browse/HBASE-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8483: - Priority: Critical (was: Minor) Fix Version/s: 0.95.1 Lets fix. You have a workaround [~ericyu] > HConnectionManager can leak ZooKeeper connections when using > deleteStaleConnection > -- > > Key: HBASE-8483 > URL: https://issues.apache.org/jira/browse/HBASE-8483 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.4 >Reporter: Eric Yu >Priority: Critical > Fix For: 0.95.1 > > Attachments: LeakZKConnections.java > > > If one thread calls deleteStaleConnection while other threads are using > connection, can leak ZK connections. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648019#comment-13648019 ] Gary Helmling commented on HBASE-8214: -- [~saint@gmail.com] That's okay by me. I'm having trouble building an assembly with trunk (not sure I know how), so it might take me a while to test with security. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, > 8214v4.txt, 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, > 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648015#comment-13648015 ] stack commented on HBASE-8214: -- Passes locally. Let me try again. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, > 8214v4.txt, 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, > 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8214: - Attachment: 8214v15.txt > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, > 8214v4.txt, 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, > 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648007#comment-13648007
]
Jonathan Hsieh commented on HBASE-8478:
---
bq. As a side note, it seems that it is time we start running the tests with
hadoop1 and hadoop2 in pre-commits.
There was a discussion on the mailing list about this. We seemed to reach a
consensus to get a green build first and then change the precommit to default
to hadoop2-only. (0.95 build would remain hadoop1 and cover that case). We
have a few sometimes hanging, sometimes failnig tests left to knock out.
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648006#comment-13648006
]
stack commented on HBASE-8478:
--
+1 on patch.
yeah, we compile against h1 and h2 on precommit but man, running full suite for
both will take forever i suppose that is what machines are for.
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648003#comment-13648003
]
Jonathan Hsieh commented on HBASE-8478:
---
Also, can you add a message for the asserts for when it fails?
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648000#comment-13648000
]
Jonathan Hsieh commented on HBASE-8478:
---
+1, from test results, lgtm.
Mind adding a more specific comment here [1] about which versions (hadoop2
throws FNFE but hadoop1 does...)? In some future time when we remove hadoop1
support it will be easier to hunt this down.
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java#L1401
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8465) Auto-drop rollback snapshot for snapshot restore
[
https://issues.apache.org/jira/browse/HBASE-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647997#comment-13647997
]
Jonathan Hsieh commented on HBASE-8465:
---
If we only have opinions let's look to how other systems do this as a tie
breaker. What happens when you take do an overwriting snapshot restore of
postgres, mysql or oracle databases.
> Auto-drop rollback snapshot for snapshot restore
>
>
> Key: HBASE-8465
> URL: https://issues.apache.org/jira/browse/HBASE-8465
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8465-trunk-v1.txt, 8465-trunk-v2.txt
>
>
> Below is an excerpt from snapshot restore javadoc:
> {code}
>* Restore the specified snapshot on the original table. (The table must be
> disabled)
>* Before restoring the table, a new snapshot with the current table state
> is created.
>* In case of failure, the table will be rolled back to the its original
> state.
> {code}
> We can improve the handling of rollbackSnapshot in two ways:
> 1. give better name to the rollbackSnapshot (adding
> {code}'-for-rollback-'{code}). Currently the name is of the form:
> String rollbackSnapshot = snapshotName + "-" +
> EnvironmentEdgeManager.currentTimeMillis();
> 2. drop rollbackSnapshot at the end of restoreSnapshot() if the restore is
> successful. We can introduce new config param, named
> 'hbase.snapshot.restore.drop.rollback', to keep compatibility with current
> behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8466) Netty messages in the logs
[
https://issues.apache.org/jira/browse/HBASE-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647982#comment-13647982
]
stack commented on HBASE-8466:
--
TestHBaseFsck should be fixed now. +1 on patch
> Netty messages in the logs
> --
>
> Key: HBASE-8466
> URL: https://issues.apache.org/jira/browse/HBASE-8466
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.0, 0.95.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Minor
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8466.v1.patch, 8466.v2.patch
>
>
> We've got this:
> {noformat}
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] OPEN
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] BOUND: 0.0.0.0/0.0.0.0:37250
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] CONNECTED:
> /226.1.1.3:60100
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] WRITTEN_AMOUNT: 129
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] DISCONNECTED
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] UNBOUND
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] CLOSED
> {noformat}
> We can fix this by adding an upstream handler that discards the message
> without printing them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647981#comment-13647981
]
stack commented on HBASE-8482:
--
[~enis] Yes. @nkeywal is the man for where to use EEM and where not. Need to
follow his reciepe.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[
https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647980#comment-13647980
]
Hadoop QA commented on HBASE-8214:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581609/8214v15.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 123
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5543//console
This message is automatically generated.
> Remove proxy and engine, rely directly on pb generated Service
> --
>
> Key: HBASE-8214
> URL: https://issues.apache.org/jira/browse/HBASE-8214
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt,
> 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt,
> 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt,
> 8214v9.txt
>
>
> Attached patch is not done. Removes two to three layers -- depending on how
> you count -- between client and rpc client and similar on server side
> (between rpc and server implementation). Strips ProtobufRpcServer/Client and
> HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8471) Server-side, remove convertion from pb type to client type before we call method
[ https://issues.apache.org/jira/browse/HBASE-8471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647975#comment-13647975 ] stack commented on HBASE-8471: -- [~anoop.hbase] Could CPs take Cells or a CellScanner? Might have to be an Iterable object. See CellUtil and how it is used in ipc. If there is only one Result going back, then could be simple Iterable that you pass in and get back. If cp wants to change what is Iterated, it can or just do pass-through. > Server-side, remove convertion from pb type to client type before we call > method > > > Key: HBASE-8471 > URL: https://issues.apache.org/jira/browse/HBASE-8471 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, regionserver >Reporter: stack > > In the regionserver, when the rpc receives a call, the call is described > using protobufs. Before we make the server-side invocation, we do a > transform on the pb param objects to make a native pojo -- e.g. from a pb > Puts into an hbase o.a.h.h.client.Put -- and only then do we make the call > against the server. > On the way out, similar, before putting the result on the wire, we will do a > convertion from o.a.h.h.client.Result into pb Result. > This issue is about our first INVESTIGATING if it is possible to do away w/ > this marshalling/unmarshalling serverside especially given the pb objects > themselves are rich in accessor and getters, etc. If it is possible to do w/ > pbs alone serverside, then we should go ahead and rip out all the serverside > convertions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647974#comment-13647974
]
Jimmy Xiang commented on HBASE-8314:
We do have a warning saying this should not happen, per nkeywal's request.
However, according to HBASE-8389, HBASE-7878 may not always work.
> HLogSplitter can retry to open a 0-length hlog file
> ---
>
> Key: HBASE-8314
> URL: https://issues.apache.org/jira/browse/HBASE-8314
> Project: HBase
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.98.0, 0.95.1
>
> Attachments: region-server.log, trunk-8314.patch, trunk-8314_v2.patch
>
>
> In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
> fail to open it since it can get the file length, therefore, master can't
> start.
> {noformat}
> java.io.IOException: Cannot obtain block length for LocatedBlock{...;
> getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:117)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Attachment: 8430v2.txt Thanks for review [~sershe] Yeah, was broke. Hopefully this version is less silly (Throw IOE now instead of crazy special hbase exception). > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt, 8430v2.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8471) Server-side, remove convertion from pb type to client type before we call method
[ https://issues.apache.org/jira/browse/HBASE-8471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647965#comment-13647965 ] Andrew Purtell commented on HBASE-8471: --- I don't know of an existing use case that manipulates a Result or the list of Results in a CP hook, but to afford that possibility is why those hooks take that argument. Substituting PB objects for POJOs will be fine as long as they can be manipulated within the hook to change what is being passed back to the client. > Server-side, remove convertion from pb type to client type before we call > method > > > Key: HBASE-8471 > URL: https://issues.apache.org/jira/browse/HBASE-8471 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, regionserver >Reporter: stack > > In the regionserver, when the rpc receives a call, the call is described > using protobufs. Before we make the server-side invocation, we do a > transform on the pb param objects to make a native pojo -- e.g. from a pb > Puts into an hbase o.a.h.h.client.Put -- and only then do we make the call > against the server. > On the way out, similar, before putting the result on the wire, we will do a > convertion from o.a.h.h.client.Result into pb Result. > This issue is about our first INVESTIGATING if it is possible to do away w/ > this marshalling/unmarshalling serverside especially given the pb objects > themselves are rich in accessor and getters, etc. If it is possible to do w/ > pbs alone serverside, then we should go ahead and rip out all the serverside > convertions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647963#comment-13647963
]
Enis Soztutar commented on HBASE-8314:
--
Sorry to come late to this. After 7878, we should never be in a situation where
this patch is needed. Otherwise, it means that we can get a file length that is
shorter than the actual file length and lose data. Lease recovery triggers, two
things lease recovery and block recovery. Shall we do an addendum to log a
warning that this should not happen on the retry?
> HLogSplitter can retry to open a 0-length hlog file
> ---
>
> Key: HBASE-8314
> URL: https://issues.apache.org/jira/browse/HBASE-8314
> Project: HBase
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.98.0, 0.95.1
>
> Attachments: region-server.log, trunk-8314.patch, trunk-8314_v2.patch
>
>
> In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
> fail to open it since it can get the file length, therefore, master can't
> start.
> {noformat}
> java.io.IOException: Cannot obtain block length for LocatedBlock{...;
> getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:117)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647957#comment-13647957
]
Enis Soztutar commented on HBASE-8482:
--
I fear that some time later we will just see that we are using
system.currentTimeMilis() there and change it again to use EEM. Unfortunately I
do not have a clever solution to change the chore + sleeper thing.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8346) Prefetching .META. rows in case only when useCache is set to true
[
https://issues.apache.org/jira/browse/HBASE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8346:
--
Fix Version/s: 0.95.1
0.98.0
Integrated to 0.95 and trunk.
Thanks for the patch, Himanshu.
Thanks for the review, Sergey.
> Prefetching .META. rows in case only when useCache is set to true
> -
>
> Key: HBASE-8346
> URL: https://issues.apache.org/jira/browse/HBASE-8346
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.95.0
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
>Priority: Minor
> Fix For: 0.98.0, 0.95.1
>
> Attachments: HBase-8346-v1.patch, HBase-8346-v2.patch,
> HBase-8346-v3.patch
>
>
> While doing a .META. lookup (HCM#locateRegionInMeta), we also prefetch some
> other region's info for that table. The usual call to the meta lookup has
> useCache variable set to true.
> Currently, it calls preFetch irrespective of the value useCache flag:
> {code}
> if (Bytes.equals(parentTable, HConstants.META_TABLE_NAME) &&
> (getRegionCachePrefetch(tableName))) {
> prefetchRegionCache(tableName, row);
> }
> {code}
> Later on, if useCache flag is set to false, it deletes the entry for that row
> from the cache with a forceDeleteCachedLocation() call. This always results
> in two calls to the .META. table in this case. The useCache variable is set
> to false in case we are retrying to find a region (regionserver failover).
> It can be verified from the log statements of a client while having a
> regionserver failover. In the below example, the client was connected to
> a1217, when a1217 got killed. The region in question is moved to a1215.
> Client got this info from META scan, where as client cache this info from
> META, but then delete it from cache as it want the latest info.
> The result is even the meta provides the latest info, it is still deleted
> This causes even the latest info to be deleted. Thus, client deletes
> a1215.abc.com even though it is correct info.
> {code}
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1217.abc.com:40020
> 13/04/15 09:49:12 WARN client.ServerCallable: Received exception, tries=1,
> numRetries=30 message=Connection refused
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed all cached region locations that map to
> a1217.abc.com,40020,1365621947381
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Scanning .META. starting at
> row=t,user7225973201630273569,00 for max=10 rows using
> hconnection-0x7786df0f
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed a1215.abc.com:40020 as a location of
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. for
> tableName=t from cache
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 WARN client.ServerCallable: Received exception, tries=2,
> numRetries=30
> message=org.apache.hadoop.hbase.exceptions.UnknownScannerException: Name:
> -6313340536390503703, already closed?
> 13/04/15 09:49:13 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 'user760712450403198900'
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/softwar
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647946#comment-13647946 ] stack commented on HBASE-8214: -- [~ghelmling] Thanks. I have not tested w/ security enabled. I have fixed a few auth checking issues that came up in the last few days making sure we present right Interface and service names. I've not set up security so not sure I know how. Mind if I fixed security issues in another issue? This one fat enough already (smile). > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, > 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, > 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647943#comment-13647943 ] Gary Helmling commented on HBASE-8214: -- I just took a final pass through 8214v15. +1 from me. Have you tested with security enabled? I don't see anything that would break it, but would be a useful gut check. I can try to setup in pseudo-distributed mode to test, but might take a little while. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, > 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, > 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8486) IS_ROOT isnt needed in HTableDescriptor.
Elliott Clark created HBASE-8486: Summary: IS_ROOT isnt needed in HTableDescriptor. Key: HBASE-8486 URL: https://issues.apache.org/jira/browse/HBASE-8486 Project: HBase Issue Type: Bug Reporter: Elliott Clark Priority: Minor HTableDescriptor has several vestigial references to the root region. We should clean these up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647936#comment-13647936
]
stack commented on HBASE-8482:
--
[~enis] The two chores were spinning because the outer chore check of
currentTimeMillis was using the EEM but the sleep was not (so we'd never
sleep). I suppose it would pass sometimes on a particular architecture but not
local here on my desktop (Thanks [~anoop.hbase])
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8485) Retry to open a HLog on more exceptions
[ https://issues.apache.org/jira/browse/HBASE-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8485: --- Status: Patch Available (was: Open) > Retry to open a HLog on more exceptions > > > Key: HBASE-8485 > URL: https://issues.apache.org/jira/browse/HBASE-8485 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Attachments: trunk-8485.patch > > > Currently we only retry to open a HLog file in case "Cannot obtain block > length" (HBASE-8314). We can retry also in case "Could not obtain the last > block locations.", "Blocklist for " + src + " has changed!", which are > possible IOException messages I can find in case open a file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8485) Retry to open a HLog on more exceptions
[ https://issues.apache.org/jira/browse/HBASE-8485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-8485: --- Attachment: trunk-8485.patch > Retry to open a HLog on more exceptions > > > Key: HBASE-8485 > URL: https://issues.apache.org/jira/browse/HBASE-8485 > Project: HBase > Issue Type: Improvement >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Attachments: trunk-8485.patch > > > Currently we only retry to open a HLog file in case "Cannot obtain block > length" (HBASE-8314). We can retry also in case "Could not obtain the last > block locations.", "Blocklist for " + src + " has changed!", which are > possible IOException messages I can find in case open a file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8346) Prefetching .META. rows in case only when useCache is set to true
[
https://issues.apache.org/jira/browse/HBASE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647930#comment-13647930
]
Sergey Shelukhin commented on HBASE-8346:
-
Ah, I see. I thought some simpler refactoring would be possible where we only
have one call to getCachedLocation.
I think v2 should also be good. Thanks
> Prefetching .META. rows in case only when useCache is set to true
> -
>
> Key: HBASE-8346
> URL: https://issues.apache.org/jira/browse/HBASE-8346
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.95.0
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
>Priority: Minor
> Attachments: HBase-8346-v1.patch, HBase-8346-v2.patch,
> HBase-8346-v3.patch
>
>
> While doing a .META. lookup (HCM#locateRegionInMeta), we also prefetch some
> other region's info for that table. The usual call to the meta lookup has
> useCache variable set to true.
> Currently, it calls preFetch irrespective of the value useCache flag:
> {code}
> if (Bytes.equals(parentTable, HConstants.META_TABLE_NAME) &&
> (getRegionCachePrefetch(tableName))) {
> prefetchRegionCache(tableName, row);
> }
> {code}
> Later on, if useCache flag is set to false, it deletes the entry for that row
> from the cache with a forceDeleteCachedLocation() call. This always results
> in two calls to the .META. table in this case. The useCache variable is set
> to false in case we are retrying to find a region (regionserver failover).
> It can be verified from the log statements of a client while having a
> regionserver failover. In the below example, the client was connected to
> a1217, when a1217 got killed. The region in question is moved to a1215.
> Client got this info from META scan, where as client cache this info from
> META, but then delete it from cache as it want the latest info.
> The result is even the meta provides the latest info, it is still deleted
> This causes even the latest info to be deleted. Thus, client deletes
> a1215.abc.com even though it is correct info.
> {code}
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1217.abc.com:40020
> 13/04/15 09:49:12 WARN client.ServerCallable: Received exception, tries=1,
> numRetries=30 message=Connection refused
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed all cached region locations that map to
> a1217.abc.com,40020,1365621947381
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Scanning .META. starting at
> row=t,user7225973201630273569,00 for max=10 rows using
> hconnection-0x7786df0f
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed a1215.abc.com:40020 as a location of
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. for
> tableName=t from cache
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 WARN client.ServerCallable: Received exception, tries=2,
> numRetries=30
> message=org.apache.hadoop.hbase.exceptions.UnknownScannerException: Name:
> -6313340536390503703, already closed?
> 13/04/15 09:49:13 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 'user760712450403198900'
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647929#comment-13647929
]
Enis Soztutar commented on HBASE-8482:
--
BTW, Anoop, your analysis is right, we just inject an IncrementingEEM to
control the time to expire the lock.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647928#comment-13647928
]
Enis Soztutar commented on HBASE-8482:
--
Wow, that is a lot of calles to currentTimeMilis, even if we are not using the
EEM, we should not be doing that many calls right?
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8346) Prefetching .META. rows in case only when useCache is set to true
[
https://issues.apache.org/jira/browse/HBASE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647922#comment-13647922
]
Hadoop QA commented on HBASE-8346:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581570/HBase-8346-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5542//console
This message is automatically generated.
> Prefetching .META. rows in case only when useCache is set to true
> -
>
> Key: HBASE-8346
> URL: https://issues.apache.org/jira/browse/HBASE-8346
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.95.0
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
>Priority: Minor
> Attachments: HBase-8346-v1.patch, HBase-8346-v2.patch,
> HBase-8346-v3.patch
>
>
> While doing a .META. lookup (HCM#locateRegionInMeta), we also prefetch some
> other region's info for that table. The usual call to the meta lookup has
> useCache variable set to true.
> Currently, it calls preFetch irrespective of the value useCache flag:
> {code}
> if (Bytes.equals(parentTable, HConstants.META_TABLE_NAME) &&
> (getRegionCachePrefetch(tableName))) {
> prefetchRegionCache(tableName, row);
> }
> {code}
> Later on, if useCache flag is set to false, it deletes the entry for that row
> from the cache with a forceDeleteCachedLocation() call. This always results
> in two calls to the .META. table in this case. The useCache variable is set
> to false in case we are retrying to find a region (regionserver failover).
> It can be verified from the log statements of a client while having a
> regionserver failover. In the below example, the client was connected to
> a1217, when a1217 got killed. The region in question is moved to a1215.
> Client got this info from META scan, where as client cache this info from
> META, but then delete it from cache as it want the latest info.
> The result is even the meta provides the latest info, it is still deleted
> This causes even the latest info to be deleted. Thus, client
[jira] [Commented] (HBASE-8389) HBASE-8354 forces Namenode into loop with lease recovery requests
[ https://issues.apache.org/jira/browse/HBASE-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647915#comment-13647915 ] Varun Sharma commented on HBASE-8389: - No, it does not. We primarily fix it by fixing it in HDFS since thats where the core of the problem is (slow recovery). > HBASE-8354 forces Namenode into loop with lease recovery requests > - > > Key: HBASE-8389 > URL: https://issues.apache.org/jira/browse/HBASE-8389 > Project: HBase > Issue Type: Bug >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.94.8 > > Attachments: 8389-0.94.txt, 8389-0.94-v2.txt, 8389-0.94-v3.txt, > 8389-0.94-v4.txt, 8389-0.94-v5.txt, 8389-0.94-v6.txt, 8389-trunk-v1.txt, > 8389-trunk-v2.patch, 8389-trunk-v2.txt, 8389-trunk-v3.txt, nn1.log, nn.log, > sample.patch > > > We ran hbase 0.94.3 patched with 8354 and observed too many outstanding lease > recoveries because of the short retry interval of 1 second between lease > recoveries. > The namenode gets into the following loop: > 1) Receives lease recovery request and initiates recovery choosing a primary > datanode every second > 2) A lease recovery is successful and the namenode tries to commit the block > under recovery as finalized - this takes < 10 seconds in our environment > since we run with tight HDFS socket timeouts. > 3) At step 2), there is a more recent recovery enqueued because of the > aggressive retries. This causes the committed block to get preempted and we > enter a vicious cycle > So we do, --> --> > > This loop is paused after 300 seconds which is the > "hbase.lease.recovery.timeout". Hence the MTTR we are observing is 5 minutes > which is terrible. Our ZK session timeout is 30 seconds and HDFS stale node > detection timeout is 20 seconds. > Note that before the patch, we do not call recoverLease so aggressively - > also it seems that the HDFS namenode is pretty dumb in that it keeps > initiating new recoveries for every call. Before the patch, we call > recoverLease, assume that the block was recovered, try to get the file, it > has zero length since its under recovery, we fail the task and retry until we > get a non zero length. So things just work. > Fixes: > 1) Expecting recovery to occur within 1 second is too aggressive. We need to > have a more generous timeout. The timeout needs to be configurable since > typically, the recovery takes as much time as the DFS timeouts. The primary > datanode doing the recovery tries to reconcile the blocks and hits the > timeouts when it tries to contact the dead node. So the recovery is as fast > as the HDFS timeouts. > 2) We have another issue I report in HDFS 4721. The Namenode chooses the > stale datanode to perform the recovery (since its still alive). Hence the > first recovery request is bound to fail. So if we want a tight MTTR, we > either need something like HDFS 4721 or we need something like this > recoverLease(...) > sleep(1000) > recoverLease(...) > sleep(configuredTimeout) > recoverLease(...) > sleep(configuredTimeout) > Where configuredTimeout should be large enough to let the recovery happen but > the first timeout is short so that we get past the moot recovery in step #1. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8214: - Attachment: 8214v15.txt The TestHBaseFsck fix was just committed -- unrelated to this patch -- and the TestIOFencing seems unrelated (passes locally). Let me retry hadoopqa. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, > 8214v5.txt, 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, > 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8482:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed. Should help w/ a test that has been failing pretty frequently of
late.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8449) Refactor recoverLease retries and pauses informed by findings over in hbase-8389
[ https://issues.apache.org/jira/browse/HBASE-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647905#comment-13647905 ] stack commented on HBASE-8449: -- Thinking on it, we could test recoverLease result. If false, wait one second or two and then retry (for the case where primary node is up, just taking its time). If it comes back false on second invocation, then we wait what we think is the hdfs-side read timeout, dfs.socket.timeout, 'public static int READ_TIMEOUT = 60 * 1000;' or some good portion of it and then leave the loop w/o redoing recoverLease. The read will likely fail but we have retrying going on around it (and Jimmy justed improved it over in hbase-8314). The amount of time to wait the second time should probably be configurable since no way for us to know the hdfs configs (Talking w/ Elliott, we should have the master ask the NN and then have it publish the important configs for regionservers to pick up in zk: TODO). We can reuse the config added by hbase-8389 and default it to 60 seconds rather than the 4 it is currently set to. In another issue, we'd add looking for isFileClosed and if it returns before the 60 seconds expires, stop waiting and retry recoverLease. > Refactor recoverLease retries and pauses informed by findings over in > hbase-8389 > > > Key: HBASE-8449 > URL: https://issues.apache.org/jira/browse/HBASE-8449 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration >Affects Versions: 0.94.7, 0.95.0 >Reporter: stack >Priority: Critical > Fix For: 0.95.1 > > > HBASE-8359 is an interesting issue that roams near and far. This issue is > about making use of the findings handily summarized on the end of hbase-8359 > which have it that trunk needs refactor around how it does its recoverLease > handling (and that the patch committed against HBASE-8359 is not what we want > going forward). > This issue is about making a patch that adds a lag between recoverLease > invocations where the lag is related to dfs timeouts -- the hdfs-side dfs > timeout -- and optionally makes use of the isFileClosed API if it is > available (a facility that is not yet committed to a branch near you and > unlikely to be within your locality with a good while to come). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8485) Retry to open a HLog on more exceptions
Jimmy Xiang created HBASE-8485: -- Summary: Retry to open a HLog on more exceptions Key: HBASE-8485 URL: https://issues.apache.org/jira/browse/HBASE-8485 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Currently we only retry to open a HLog file in case "Cannot obtain block length" (HBASE-8314). We can retry also in case "Could not obtain the last block locations.", "Blocklist for " + src + " has changed!", which are possible IOException messages I can find in case open a file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8449) Refactor recoverLease retries and pauses informed by findings over in hbase-8389
[ https://issues.apache.org/jira/browse/HBASE-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8449: - Description: HBASE-8359 is an interesting issue that roams near and far. This issue is about making use of the findings handily summarized on the end of hbase-8359 which have it that trunk needs refactor around how it does its recoverLease handling (and that the patch committed against HBASE-8359 is not what we want going forward). This issue is about making a patch that adds a lag between recoverLease invocations where the lag is related to dfs timeouts -- the hdfs-side dfs timeout -- and optionally makes use of the isFileClosed API if it is available (a facility that is not yet committed to a branch near you and unlikely to be within your locality with a good while to come). was: HBASE-8354 is an interesting issue that roams near and far. This issue is about making use of the findings handily summarized on the end of hbase-8354 which have it that trunk needs refactor around how it does its recoverLease handling (and that the patch committed against HBASE-8354 is not what we want going forward). This issue is about making a patch that adds a lag between recoverLease invocations where the lag is related to dfs timeouts -- the hdfs-side dfs timeout -- and optionally makes use of the isFileClosed API if it is available (a facility that is not yet committed to a branch near you and unlikely to be within your locality with a good while to come). > Refactor recoverLease retries and pauses informed by findings over in > hbase-8389 > > > Key: HBASE-8449 > URL: https://issues.apache.org/jira/browse/HBASE-8449 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration >Affects Versions: 0.94.7, 0.95.0 >Reporter: stack >Priority: Critical > Fix For: 0.95.1 > > > HBASE-8359 is an interesting issue that roams near and far. This issue is > about making use of the findings handily summarized on the end of hbase-8359 > which have it that trunk needs refactor around how it does its recoverLease > handling (and that the patch committed against HBASE-8359 is not what we want > going forward). > This issue is about making a patch that adds a lag between recoverLease > invocations where the lag is related to dfs timeouts -- the hdfs-side dfs > timeout -- and optionally makes use of the isFileClosed API if it is > available (a facility that is not yet committed to a branch near you and > unlikely to be within your locality with a good while to come). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8314) HLogSplitter can retry to open a 0-length hlog file
[
https://issues.apache.org/jira/browse/HBASE-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647892#comment-13647892
]
stack commented on HBASE-8314:
--
+1 on patch. What @nkeywal said above.
> HLogSplitter can retry to open a 0-length hlog file
> ---
>
> Key: HBASE-8314
> URL: https://issues.apache.org/jira/browse/HBASE-8314
> Project: HBase
> Issue Type: Bug
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.98.0, 0.95.1
>
> Attachments: region-server.log, trunk-8314.patch, trunk-8314_v2.patch
>
>
> In case a HLog file is of size 0, and it is under recovery, HLogSplitter will
> fail to open it since it can get the file length, therefore, master can't
> start.
> {noformat}
> java.io.IOException: Cannot obtain block length for LocatedBlock{...;
> getBlockSize()=0; corrupt=false; offset=0; locs=[...]}
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:238)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:182)
> at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:124)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:117)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1080)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8389) HBASE-8354 forces Namenode into loop with lease recovery requests
[ https://issues.apache.org/jira/browse/HBASE-8389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647885#comment-13647885 ] Jimmy Xiang commented on HBASE-8389: [~varunsharma], when you played with the patch for this jira, does your HBase have HBASE-8314 and HBASE-8321? I think HBASE-8314 may give you extra time for the file to recover. > HBASE-8354 forces Namenode into loop with lease recovery requests > - > > Key: HBASE-8389 > URL: https://issues.apache.org/jira/browse/HBASE-8389 > Project: HBase > Issue Type: Bug >Reporter: Varun Sharma >Assignee: Varun Sharma >Priority: Critical > Fix For: 0.94.8 > > Attachments: 8389-0.94.txt, 8389-0.94-v2.txt, 8389-0.94-v3.txt, > 8389-0.94-v4.txt, 8389-0.94-v5.txt, 8389-0.94-v6.txt, 8389-trunk-v1.txt, > 8389-trunk-v2.patch, 8389-trunk-v2.txt, 8389-trunk-v3.txt, nn1.log, nn.log, > sample.patch > > > We ran hbase 0.94.3 patched with 8354 and observed too many outstanding lease > recoveries because of the short retry interval of 1 second between lease > recoveries. > The namenode gets into the following loop: > 1) Receives lease recovery request and initiates recovery choosing a primary > datanode every second > 2) A lease recovery is successful and the namenode tries to commit the block > under recovery as finalized - this takes < 10 seconds in our environment > since we run with tight HDFS socket timeouts. > 3) At step 2), there is a more recent recovery enqueued because of the > aggressive retries. This causes the committed block to get preempted and we > enter a vicious cycle > So we do, --> --> > > This loop is paused after 300 seconds which is the > "hbase.lease.recovery.timeout". Hence the MTTR we are observing is 5 minutes > which is terrible. Our ZK session timeout is 30 seconds and HDFS stale node > detection timeout is 20 seconds. > Note that before the patch, we do not call recoverLease so aggressively - > also it seems that the HDFS namenode is pretty dumb in that it keeps > initiating new recoveries for every call. Before the patch, we call > recoverLease, assume that the block was recovered, try to get the file, it > has zero length since its under recovery, we fail the task and retry until we > get a non zero length. So things just work. > Fixes: > 1) Expecting recovery to occur within 1 second is too aggressive. We need to > have a more generous timeout. The timeout needs to be configurable since > typically, the recovery takes as much time as the DFS timeouts. The primary > datanode doing the recovery tries to reconcile the blocks and hits the > timeouts when it tries to contact the dead node. So the recovery is as fast > as the HDFS timeouts. > 2) We have another issue I report in HDFS 4721. The Namenode chooses the > stale datanode to perform the recovery (since its still alive). Hence the > first recovery request is bound to fail. So if we want a tight MTTR, we > either need something like HDFS 4721 or we need something like this > recoverLease(...) > sleep(1000) > recoverLease(...) > sleep(configuredTimeout) > recoverLease(...) > sleep(configuredTimeout) > Where configuredTimeout should be large enough to let the recovery happen but > the first timeout is short so that we get past the moot recovery in step #1. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647856#comment-13647856
]
Hadoop QA commented on HBASE-8482:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581568/8482.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5540//console
This message is automatically generated.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/te
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[
https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647858#comment-13647858
]
Hadoop QA commented on HBASE-8214:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12581577/8214v15.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 123
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestIOFencing
org.apache.hadoop.hbase.util.TestHBaseFsck
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5541//console
This message is automatically generated.
> Remove proxy and engine, rely directly on pb generated Service
> --
>
> Key: HBASE-8214
> URL: https://issues.apache.org/jira/browse/HBASE-8214
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: stack
>Assignee: stack
> Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt,
> 8214v12.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, 8214v5.txt,
> 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, 8214v9.txt
>
>
> Attached patch is not done. Removes two to three layers -- depending on how
> you count -- between client and rpc client and similar on server side
> (between rpc and server implementation). Strips ProtobufRpcServer/Client and
> HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647855#comment-13647855 ] Sergey Shelukhin commented on HBASE-8430: - Where is it actually thrown? Also, no changes to BaseDecorder didn't change at least > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8466) Netty messages in the logs
[
https://issues.apache.org/jira/browse/HBASE-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647847#comment-13647847
]
Elliott Clark commented on HBASE-8466:
--
The apache rat plugin had a lot of the ignores removed. There's one that was
removed but was actually needed. HBASE-8470 was just recently checked in to fix
things.
I'm +1 as long as the patch is apache-rat clean.
> Netty messages in the logs
> --
>
> Key: HBASE-8466
> URL: https://issues.apache.org/jira/browse/HBASE-8466
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.0, 0.95.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Minor
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8466.v1.patch, 8466.v2.patch
>
>
> We've got this:
> {noformat}
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] OPEN
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] BOUND: 0.0.0.0/0.0.0.0:37250
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] CONNECTED:
> /226.1.1.3:60100
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] WRITTEN_AMOUNT: 129
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] DISCONNECTED
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] UNBOUND
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] CLOSED
> {noformat}
> We can fix this by adding an upstream handler that discards the message
> without printing them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647848#comment-13647848 ] Andrew Purtell commented on HBASE-8214: --- Can't review the changes in detail due to time constraints, but I scanned for points of interest and have no concerns on the parts that touch coprocessors or security. However, has this change been tested with security/krb enabled? > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, 8214v5.txt, > 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647844#comment-13647844 ] Elliott Clark commented on HBASE-8214: -- Yeah if the expected tests pass I'm +1 this. Its much cleaner than what was there before. > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, 8214v5.txt, > 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Affects Version/s: 0.95.0 Status: Patch Available (was: Open) > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8430) Cell decoder/scanner/etc. should not hide exceptions
[ https://issues.apache.org/jira/browse/HBASE-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8430: - Attachment: 8430.txt Move HBaseIOException back into hbase-common and then have CellScanner#advance throw CellScannerAdvanceException, a subclass of HBaseIOException. Patch is big only because stuff is moved around. > Cell decoder/scanner/etc. should not hide exceptions > > > Key: HBASE-8430 > URL: https://issues.apache.org/jira/browse/HBASE-8430 > Project: HBase > Issue Type: Bug > Components: IPC/RPC, Protobufs >Reporter: Sergey Shelukhin >Assignee: stack >Priority: Critical > Fix For: 0.95.1 > > Attachments: 8430.txt > > > Cell scanner, base decoder, etc., hide IOException inside runtime exception. > This can lead to unexpected behavior because a lot of code only expects > IOException. There's no logical justification behind this hiding so it should > be removed before it's too late (the sooner we do it the less throws > declarations need to be added) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8214) Remove proxy and engine, rely directly on pb generated Service
[ https://issues.apache.org/jira/browse/HBASE-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8214: - Attachment: 8214v15.txt > Remove proxy and engine, rely directly on pb generated Service > -- > > Key: HBASE-8214 > URL: https://issues.apache.org/jira/browse/HBASE-8214 > Project: HBase > Issue Type: Bug > Components: IPC/RPC >Reporter: stack >Assignee: stack > Attachments: 8124.txt, 8214v10.txt, 8214v11.txt, 8214v12 (1).txt, > 8214v12.txt, 8214v15.txt, 8214v2.txt, 8214v3.txt, 8214v4.txt, 8214v5.txt, > 8214v6.txt, 8214v7.txt, 8214v8-nosecinfo.patch, 8214v8.txt, 8214v9.txt > > > Attached patch is not done. Removes two to three layers -- depending on how > you count -- between client and rpc client and similar on server side > (between rpc and server implementation). Strips ProtobufRpcServer/Client and > HBaseClientRpc/HBaseServerRpc. Also gets rid of proxy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647818#comment-13647818
]
stack commented on HBASE-8482:
--
[~anoop.hbase] [~nkeywal] is the one who knows best how we should use
environmentedge. It looks like the incrementing environment edge will never
work while chore sort of uses it rather than fully uses it (the backing sleeper
class is not using it so we'd never sleep).
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647803#comment-13647803
]
Anoop Sam John commented on HBASE-8482:
---
[~stack] This will help us to make the test pass. I think we had some issue
which was for converting all the time usages to use EnvironmentEdgeManager.
[~lhofhansl] was working on that I guess. So can we change this way now? Just
asking.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8346) Prefetching .META. rows in case only when useCache is set to true
[
https://issues.apache.org/jira/browse/HBASE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Vashishtha updated HBASE-8346:
---
Attachment: HBase-8346-v3.patch
This is what you meant Sergey? Thanks for taking a look.
> Prefetching .META. rows in case only when useCache is set to true
> -
>
> Key: HBASE-8346
> URL: https://issues.apache.org/jira/browse/HBASE-8346
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.95.0
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
>Priority: Minor
> Attachments: HBase-8346-v1.patch, HBase-8346-v2.patch,
> HBase-8346-v3.patch
>
>
> While doing a .META. lookup (HCM#locateRegionInMeta), we also prefetch some
> other region's info for that table. The usual call to the meta lookup has
> useCache variable set to true.
> Currently, it calls preFetch irrespective of the value useCache flag:
> {code}
> if (Bytes.equals(parentTable, HConstants.META_TABLE_NAME) &&
> (getRegionCachePrefetch(tableName))) {
> prefetchRegionCache(tableName, row);
> }
> {code}
> Later on, if useCache flag is set to false, it deletes the entry for that row
> from the cache with a forceDeleteCachedLocation() call. This always results
> in two calls to the .META. table in this case. The useCache variable is set
> to false in case we are retrying to find a region (regionserver failover).
> It can be verified from the log statements of a client while having a
> regionserver failover. In the below example, the client was connected to
> a1217, when a1217 got killed. The region in question is moved to a1215.
> Client got this info from META scan, where as client cache this info from
> META, but then delete it from cache as it want the latest info.
> The result is even the meta provides the latest info, it is still deleted
> This causes even the latest info to be deleted. Thus, client deletes
> a1215.abc.com even though it is correct info.
> {code}
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1217.abc.com:40020
> 13/04/15 09:49:12 WARN client.ServerCallable: Received exception, tries=1,
> numRetries=30 message=Connection refused
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed all cached region locations that map to
> a1217.abc.com,40020,1365621947381
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Scanning .META. starting at
> row=t,user7225973201630273569,00 for max=10 rows using
> hconnection-0x7786df0f
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed a1215.abc.com:40020 as a location of
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. for
> tableName=t from cache
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 WARN client.ServerCallable: Received exception, tries=2,
> numRetries=30
> message=org.apache.hadoop.hbase.exceptions.UnknownScannerException: Name:
> -6313340536390503703, already closed?
> 13/04/15 09:49:13 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 'user760712450403198900'
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8482:
-
Attachment: 8482.txt
Going off Anoops' prompting, I took a look at environmentedge and indeed, the
HRS periodic flush checker and the compaction checker are calling
environmentedge currentTimeMillis thousands of times a second; means we expire
locks well before their time.
Here is something that works locally. It undoes Chore calling
currentTimeMillis from environmentedge.
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8482) TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
[
https://issues.apache.org/jira/browse/HBASE-8482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8482:
-
Assignee: stack
Status: Patch Available (was: Open)
> TestHBaseFsck#testCheckTableLocks broke; java.lang.AssertionError:
> expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> ---
>
> Key: HBASE-8482
> URL: https://issues.apache.org/jira/browse/HBASE-8482
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.95.1
>
> Attachments: 8482.txt
>
>
> I've been looking into this test failure because I thought it particular to
> my rpc hackery.
> What I see is like the subject:
> {code}
> java.lang.AssertionError: expected:<[]> but was:<[EXPIRED_TABLE_LOCK]>
> {code}
> and later in same unit test:
> {code}
> java.lang.AssertionError: expected:<[EXPIRED_TABLE_LOCK]> but
> was:<[EXPIRED_TABLE_LOCK, EXPIRED_TABLE_LOCK]>
> {code}
> The test creates a write lock and then expires it. In subject failure, we
> are expiring the lock ahead of the time it should be. Easier for me to
> reproduce is that the second write lock we put in place is not allowed to
> happen because of the presence of the first lock EVEN THOUGH IT HAS BEEN
> JUDGED EXPIRED:
> {code}
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=387, purpose=testCheckTableLocks,
> isShared=false, createTime=129898749]
> 2013-05-02 00:34:42,715 INFO [Thread-183] lock.ZKInterProcessLockBase(431):
> Lock is held by: write-testing utility00
> ERROR: Table lock acquire attempt found:[tableName=foo,
> lockOwner=localhost,6,1, threadId=349, purpose=testCheckTableLocks,
> isShared=false, createTime=28506852]
> {code}
> Above, you see the expired lock and then our hbck lock visitor has it that
> the second lock is expired because it is held by the first lock.
> I can keep looking at this but input would be appreciated.
> It failed in recent trunk build
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/4090/testReport/junit/org.apache.hadoop.hbase.util/TestHBaseFsck/testCheckTableLocks/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8484) Adding diagnostic messages when the RegionServer hists an IOException in RegionServerReport
Manukranth Kolloju created HBASE-8484: - Summary: Adding diagnostic messages when the RegionServer hists an IOException in RegionServerReport Key: HBASE-8484 URL: https://issues.apache.org/jira/browse/HBASE-8484 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.89-fb Reporter: Manukranth Kolloju Priority: Trivial Fix For: 0.89-fb This diff adds instrumentation in the case when we catch IOException in regionserver report. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8478:
-
Status: Patch Available (was: Open)
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8478) HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under hadoop2 profile
[
https://issues.apache.org/jira/browse/HBASE-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8478:
-
Attachment: hbase-8478_v1.patch
This went into hadoop-2 code line (from 0.21)
{code}
HADOOP-6201. Change FileSystem::listStatus contract to throw
FileNotFoundException if the directory does not exist, rather than letting
this be implementation-specific. Contributed by Jakob Homan.
{code}
{code}
mvn test -DskipITs -Dhadoop.profile=2.0
-Dtest=TestHRegion#testRecoveredEditsReplayCompaction
-Dhadoop-two.version=2.0.4-alpha
Running org.apache.hadoop.hbase.regionserver.TestHRegion
2013-05-02 11:04:43.134 java[2435:1203] Unable to load realm info from
SCDynamicStore
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.109 sec
{code}
Attaching a simple patch.
As a side note, it seems that it is time we start running the tests with
hadoop1 and hadoop2 in pre-commits.
> HBASE-2231 breaks TestHRegion#testRecoveredEditsReplayCompaction under
> hadoop2 profile
> --
>
> Key: HBASE-8478
> URL: https://issues.apache.org/jira/browse/HBASE-8478
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction, hadoop2, Protobufs
>Affects Versions: 0.98.0, 0.95.1
>Reporter: Jonathan Hsieh
>Assignee: Enis Soztutar
> Fix For: 0.98.0
>
> Attachments: hbase-8478_v1.patch
>
>
> TestHRegion#testRecoveredEditsReplyCompaction and
> TestHRegionBusyWait#testRecoveredEditsReplyCompaction fail against the
> hadoop2 profile due to HBASE-2231.
> If you checkout at the patch on trunk, the error trace looks like this:
> {code}
> type="java.io.FileNotFoundException">java.io.FileNotFoundException: File
> /home/jon/proj/hbase/hbase-server/target/test-data/05b5be10-bc88-40b0-a274-d1ebffe24e85/TestHRegiontestRecoveredEditsReplayCompaction/testRecoveredEditsReplayCompaction/f1de1d311572557ca13c4cb810ebfc0b/.tmp
> does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:340)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1418)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1458)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:569)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testRecoveredEditsReplayCompaction(TestHRegion.java:462)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:226)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:133)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:114)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:188)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:166)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:86)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:101)
> {code}
> This was found via git bisect.
> {{git bisect run mvn clean install test -DskipITs -Dhadoop.profile=2.0
> -Dtest=TestHRegion#testRecoveredEditsReplayCompaction}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: ht
[jira] [Commented] (HBASE-8346) Prefetching .META. rows in case only when useCache is set to true
[
https://issues.apache.org/jira/browse/HBASE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647759#comment-13647759
]
Himanshu Vashishtha commented on HBASE-8346:
Sure thing. Though, getCachedLocation will return some non-null on second
invocation inside if(useCache) only after prefetchRegionCache is called.
> Prefetching .META. rows in case only when useCache is set to true
> -
>
> Key: HBASE-8346
> URL: https://issues.apache.org/jira/browse/HBASE-8346
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.95.0
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
>Priority: Minor
> Attachments: HBase-8346-v1.patch, HBase-8346-v2.patch
>
>
> While doing a .META. lookup (HCM#locateRegionInMeta), we also prefetch some
> other region's info for that table. The usual call to the meta lookup has
> useCache variable set to true.
> Currently, it calls preFetch irrespective of the value useCache flag:
> {code}
> if (Bytes.equals(parentTable, HConstants.META_TABLE_NAME) &&
> (getRegionCachePrefetch(tableName))) {
> prefetchRegionCache(tableName, row);
> }
> {code}
> Later on, if useCache flag is set to false, it deletes the entry for that row
> from the cache with a forceDeleteCachedLocation() call. This always results
> in two calls to the .META. table in this case. The useCache variable is set
> to false in case we are retrying to find a region (regionserver failover).
> It can be verified from the log statements of a client while having a
> regionserver failover. In the below example, the client was connected to
> a1217, when a1217 got killed. The region in question is moved to a1215.
> Client got this info from META scan, where as client cache this info from
> META, but then delete it from cache as it want the latest info.
> The result is even the meta provides the latest info, it is still deleted
> This causes even the latest info to be deleted. Thus, client deletes
> a1215.abc.com even though it is correct info.
> {code}
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1217.abc.com:40020
> 13/04/15 09:49:12 WARN client.ServerCallable: Received exception, tries=1,
> numRetries=30 message=Connection refused
> 13/04/15 09:49:12 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed all cached region locations that map to
> a1217.abc.com,40020,1365621947381
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Scanning .META. starting at
> row=t,user7225973201630273569,00 for max=10 rows using
> hconnection-0x7786df0f
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Removed a1215.abc.com:40020 as a location of
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. for
> tableName=t from cache
> 13/04/15 09:49:13 DEBUG client.MetaScanner: Current INFO from scan results =
> {NAME =>
> 't,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c.',
> STARTKEY => 'user7225973201630273569', ENDKEY => '', ENCODED =>
> 40382355b8c45e1338d620c018f8ff6c,}
> 13/04/15 09:49:13 DEBUG client.HConnectionManager$HConnectionImplementation:
> Cached location for
> t,user7225973201630273569,1365536809331.40382355b8c45e1338d620c018f8ff6c. is
> a1215.abc.com:40020
> 13/04/15 09:49:13 WARN client.ServerCallable: Received exception, tries=2,
> numRetries=30
> message=org.apache.hadoop.hbase.exceptions.UnknownScannerException: Name:
> -6313340536390503703, already closed?
> 13/04/15 09:49:13 DEBUG client.ClientScanner: Advancing internal scanner to
> startKey at 'user760712450403198900'
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.at
[jira] [Commented] (HBASE-8453) TestImportExport failing again due to configuration issues
[
https://issues.apache.org/jira/browse/HBASE-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647743#comment-13647743
]
Andrew Purtell commented on HBASE-8453:
---
bq. I noticed an internal ip/hostname – could this be a ec2 internal-external
ip reverse-dns problem?
That was my thought too since the failures only happened in an EC2 environment,
but it hosed a Hadoop 1 build that was otherwise passing, so I reverted the
change. Need to dump out configs before and after and eyeball differences.
> TestImportExport failing again due to configuration issues
> --
>
> Key: HBASE-8453
> URL: https://issues.apache.org/jira/browse/HBASE-8453
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Affects Versions: 0.98.0, 0.94.8, 0.95.1
>Reporter: Andrew Purtell
> Fix For: 0.98.0, 0.94.8, 0.95.1
>
> Attachments: 8453.patch, 8453-v1-0.94.patch
>
>
> TestImportExport fails for me with a connection refused exception:
> {noformat}
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:162)
> at
> org.apache.hadoop.yarn.client.YarnClientImpl.getNewApplication(YarnClientImpl.java:121)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(ResourceMgrDelegate.java:107)
> at org.apache.hadoop.mapred.YARNRunner.getNewJobID(YARNRunner.java:231)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:352)
> [...]
> Caused by: com.google.protobuf.ServiceException: java.net.ConnectException:
> Call From ip-10-174-75-236/10.174.75.236 to 0.0.0.0:8032 failed on connection
> exception: java.net.ConnectException: Connection refused; For more details
> see: http://wiki.apache.org/hadoop/ConnectionRefused
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212)
> at com.sun.proxy.$Proxy89.getNewApplication(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getNewApplication(ClientRMProtocolPBClientImpl.java:159)
> ... 42 more
> {noformat}
> Settings in the MiniMRCluster configuration are not properly propagated in
> this test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8466) Netty messages in the logs
[
https://issues.apache.org/jira/browse/HBASE-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647737#comment-13647737
]
Nicolas Liochon commented on HBASE-8466:
I was about to commit, but there is this
-1 release audit. The applied patch generated 1 release audit warnings (more
than the trunk's current 0 warnings).
It could be related to this patch? It's strange...
> Netty messages in the logs
> --
>
> Key: HBASE-8466
> URL: https://issues.apache.org/jira/browse/HBASE-8466
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.98.0, 0.95.0
>Reporter: Nicolas Liochon
>Assignee: Nicolas Liochon
>Priority: Minor
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 8466.v1.patch, 8466.v2.patch
>
>
> We've got this:
> {noformat}
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] OPEN
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a] BOUND: 0.0.0.0/0.0.0.0:37250
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] CONNECTED:
> /226.1.1.3:60100
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 => /226.1.1.3:60100] WRITTEN_AMOUNT: 129
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] DISCONNECTED
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] UNBOUND
> ATTENTION: The pipeline contains no upstream handlers; discarding: [id:
> 0x1f79354a, 0.0.0.0/0.0.0.0:37250 :> /226.1.1.3:60100] CLOSED
> {noformat}
> We can fix this by adding an upstream handler that discards the message
> without printing them.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8469) [hadoop2] Several tests break because of HDFS-4305
[
https://issues.apache.org/jira/browse/HBASE-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647734#comment-13647734
]
Hudson commented on HBASE-8469:
---
Integrated in hbase-0.95 #177 (See
[https://builds.apache.org/job/hbase-0.95/177/])
HBASE-8469 [hadoop2] Several tests break because of HDFS-4305 (Revision
1478369)
Result = FAILURE
jmhsieh :
Files :
* /hbase/branches/0.95/hbase-server/src/test/resources/hdfs-site.xml
> [hadoop2] Several tests break because of HDFS-4305
> --
>
> Key: HBASE-8469
> URL: https://issues.apache.org/jira/browse/HBASE-8469
> Project: HBase
> Issue Type: Sub-task
> Components: test
>Affects Versions: 0.98.0, 0.94.6.1, 0.95.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 0.98.0, 0.95.1
>
> Attachments: hbase-8469.94.patch, hbase-8469.patch,
> hbase-8469.v2.patch, hbase-8469.v3.patch
>
>
> Several unit tests will break due to HDFS-4305 (which enforces a min block
> size) because apprently, we set our block size smaller in these tests.
> {code}
> Specified block size is less than configured minimum value
> (dfs.namenode.fs-limits.min-block-size): 1024 < 1048576
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:1816)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1795)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:418)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:205)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44134)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1696)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1690)
> {code}
> org.apache.hadoop.hbase.regionserver.TestHRegion.testgetHDFSBlocksDistribution
> org.apache.hadoop.hbase.regionserver.TestHRegionBusyWait.testgetHDFSBlocksDistribution
> org.apache.hadoop.hbase.replication.TestMasterReplication.testCyclicReplication
> org.apache.hadoop.hbase.replication.TestMasterReplication.testSimplePutDelete
> org.apache.hadoop.hbase.replication.TestMultiSlaveReplication.testMultiSlaveReplication
> org.apache.hadoop.hbase.util.TestFSUtils.testcomputeHDFSBlocksDistribution
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3787) Increment is non-idempotent but client retries RPC
[ https://issues.apache.org/jira/browse/HBASE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-3787: Status: Patch Available (was: Open) > Increment is non-idempotent but client retries RPC > -- > > Key: HBASE-3787 > URL: https://issues.apache.org/jira/browse/HBASE-3787 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 0.94.4, 0.95.2 >Reporter: dhruba borthakur >Assignee: Sergey Shelukhin >Priority: Critical > Fix For: 0.95.1 > > Attachments: HBASE-3787-partial.patch, HBASE-3787-v0.patch > > > The HTable.increment() operation is non-idempotent. The client retries the > increment RPC a few times (as specified by configuration) before throwing an > error to the application. This makes it possible that the same increment call > be applied twice at the server. > For increment operations, is it better to use > HConnectionManager.getRegionServerWithoutRetries()? Another option would be > to enhance the IPC module to make the RPC server correctly identify if the > RPC is a retry attempt and handle accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
