[jira] [Created] (HBASE-22198) Fix flakey TestAsyncTableGetMultiThreaded
Duo Zhang created HBASE-22198: - Summary: Fix flakey TestAsyncTableGetMultiThreaded Key: HBASE-22198 URL: https://issues.apache.org/jira/browse/HBASE-22198 Project: HBase Issue Type: Bug Reporter: Duo Zhang https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2959/testReport/junit/org.apache.hadoop.hbase.client/TestAsyncTableGetMultiThreaded/test/ The error is thrown from an admin method, where we do not have any retries if the region is not online yet. Should be a test issue, let me fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-21886) Run ITBLL for branch-2.2
[ https://issues.apache.org/jira/browse/HBASE-21886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang resolved HBASE-21886. Resolution: Fixed > Run ITBLL for branch-2.2 > > > Key: HBASE-21886 > URL: https://issues.apache.org/jira/browse/HBASE-21886 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 2.2.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22197) document delegation tokens
Sean Busbey created HBASE-22197: --- Summary: document delegation tokens Key: HBASE-22197 URL: https://issues.apache.org/jira/browse/HBASE-22197 Project: HBase Issue Type: Task Components: documentation, security Reporter: Sean Busbey Got some questions today about delegation tokens. The asker referenced this very old page from the old HBase wiki (which we deprecated as defunct ages ago): https://wiki.apache.org/hadoop/Hbase/HBaseTokenAuthentication I was going to refer them to the up to date docs, since I thought we migrated all the content. However, a review of the current Ref Guide shows that we do not have docs on delegation tokens. Ideally this would be 2 sections: * an addition to the "secure configuration" section that explained how to use delegation tokens in a secure cluster, with the primary example of their use in MR and/or spark * an appendix that explains the current implementation for those working on hbase internals -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19894) Add seePastDeleteMarkers() option in scan
[ https://issues.apache.org/jira/browse/HBASE-19894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal resolved HBASE-19894. --- Resolution: Won't Fix > Add seePastDeleteMarkers() option in scan > - > > Key: HBASE-19894 > URL: https://issues.apache.org/jira/browse/HBASE-19894 > Project: HBase > Issue Type: Sub-task > Components: scan >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19826) Provide a option to see rows behind a delete in a time range queries
[
https://issues.apache.org/jira/browse/HBASE-19826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ankit Singhal resolved HBASE-19826.
---
Resolution: Fixed
Fix Version/s: 2.0.0
> Provide a option to see rows behind a delete in a time range queries
>
>
> Key: HBASE-19826
> URL: https://issues.apache.org/jira/browse/HBASE-19826
> Project: HBase
> Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 2.0.0
>
>
> We can provide an option (something like seePastDeleteMarkers) in a scan to
> let the user see the versions behind the delete marker even if
> keepDeletedCells is set to false in the descriptor.
> With the previous version, we workaround the same in preStoreScannerOpen
> hook. For reference PHOENIX-4277
> {code}
> @Override
> public KeyValueScanner preStoreScannerOpen(final
> ObserverContext c,
> final Store store, final Scan scan, final NavigableSet
> targetCols,
> final KeyValueScanner s) throws IOException {
>
> if (scan.isRaw() ||
> ScanInfoUtil.isKeepDeletedCells(store.getScanInfo()) ||
> scan.getTimeRange().getMax() == HConstants.LATEST_TIMESTAMP ||
> TransactionUtil.isTransactionalTimestamp(scan.getTimeRange().getMax())) {
> return s;
> }
>
> ScanInfo scanInfo =
> ScanInfoUtil.cloneScanInfoWithKeepDeletedCells(store.getScanInfo());
> return new StoreScanner(store, scanInfo, scan, targetCols,
>
> c.getEnvironment().getRegion().getReadpoint(scan.getIsolationLevel()));
> }
> {code}
> Another way is to provide a way to set KEEP_DELETED_CELLS to true in
> ScanOptions of preStoreScannerOpen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: [DISCUSS] move to gitbox
+1 On 4/6/19 8:22 AM, 张铎(Duo Zhang) wrote: OK, now we have the yetus set up for GitHub PR, and I've tried to use GitHub PR to process several issues. For example HBASE-22174, Sean Busbey approved on the PR, and I used the GitHub PR to commit to master, and then cherry-picked the commit to other branches, and committed them using command line. Worked pretty fine. The only problem is that I accidentally clicked the default merge button and created a merge commit in the commit history... So I'm +1 on only allowing rebase merging, for squash merging, it is not a big deal to let the author squash the commits before merging. And at least, let's disable the default 'Create a Merge Commit' option... Thanks. Sean Busbey 于2019年1月10日周四 上午2:03写道： sure thing. let me draft it up now. On Wed, Jan 9, 2019 at 11:29 AM Andrew Purtell wrote: According to the ticket the main repo and third-party repo have been migrated. We are just waiting on site. I'd appreciate it if we can send out that email now, because I'd like to continue commiting toward 1.5.0. At least, a pointer to docs on how the integration functions and the steps a committer needs to take to push and pull changes would be appreciated. On Jan 9, 2019, at 8:56 AM, Sean Busbey wrote: I filed a ticket with INFRA: https://issues.apache.org/jira/browse/INFRA-17597 The actual transition is supposed to only take a few minutes. Once it's done we should send a NOTICE email to dev@hbase summarizing what has changed and what folks will need to do different. On 2019/01/08 19:07:30, Peter Somogyi wrote: I believe we reached consensus to migrate our remaining repositories to GitBox before the mandatory migration which will happen on 7th of February. Apart from 'hbase' we still have 'hbase-site' and 'hbase-thirdparty' repositories that also require the same migration process. From users' point of view they could still use git:// git.apache.org/hbase.git for read only access, only committers need to change the remote URL to the GitBox one. Jenkins jobs are already using the GitHub URL for cloning the repository and I created a patch for the documentation and website changes in HBASE-21685 that we can merge after the process is competed. There's still outstanding work to do before we have good guidelines on accepting pull requests on GitHub, but the GitBox migration doesn't require our committers to start working with PRs in a different way. If there is no disagreement I'd kindly ask one of the PMC members to reach out to INFRA to perform the migration. Thanks, Peter On Sun, Dec 9, 2018 at 12:56 AM Andrew Purtell < andrew.purt...@gmail.com> wrote: Sounds good to me except squash merge at commit of PR shouldn’t be an option it should be required except for good reason (and I don’t know what that would be ) On Dec 8, 2018, at 3:28 PM, Stack wrote: On Fri, Dec 7, 2018 at 6:23 PM Sean Busbey wrote: The move to gitbox doesn't require us to only accept github PRs. Given the current rate of contributions via patches on JIRA vs GitHub PRs, I wouldn't want to push for that now. The change does make it easier for us to start encouraging PR submissions, because committers will be able to directly merge from the GitHub UI. I'd recommend we try to keep this as a small incremental change. That would mean: * committers ensure there's an associated JIRA for release note and precommit checks (that can be just by pinging the contributor to go make one) * backports are still handled by the committer if they're simple and the contributor if there's a problem. I think saying "open a new PR to backport to branch FOO" is perfectly reasonable since it's analogous to when we ask contributors to attach a branch specific patch. * committers ensure the pushed commit has a message that follows our current practice (which would mean looking out for the "helpful" subject wrapping) * Squash merge is an option when the committer goes to accept the PR. the contributor is free to either push additional commits or squash on their branch when working through reviews, I don't think we need to weigh in on how contributors choose which of those works best for them. That way we can also incrementally improve how well we handle PR submissions by better documenting expectations and building up additional tooling (e.g. having our precommit feedback go directly to the PR instead of being tied to a JIRA) This seems reasonable to me. Andrew's strawman would be too radical a change. Thanks, S On Fri, Dec 7, 2018 at 12:09 PM Stack wrote: On Fri, Dec 7, 2018 at 9:03 AM Sean Busbey wrote: Hi folks! Per the email from infra "[NOTICE] Mandatory relocation of Apache git repositories on git-wip-us.apache.org" ( https://s.apache.org/0sfe ) it looks like the future of interacting directly with PRs is coming sooner rather than later. I think we should move to gitbox ASAP rather than wait for the crunch. If we hit a rough
Re: [DISCUSS] EOL for branch-1.2
+1 On 4/5/19 9:35 PM, Sean Busbey wrote: Hi folks! Back when our stable pointer first moved off of branch-1.2 releases I said I'd do ~6 months of releases[1]. I'm preparing an RC for the 1.2.12 release[2] and it's been ~6.5 months. I intend 1.2.12 to be the last release I manage off of branch-1.2. If anyone else sees a reason to keep this release line going and is willing to step in as RM please speak up. This is the template for the notice that's been in the ANNOUNCE emails for 1.2.7 - 1.2.11: All users of previous 1.2.z releases are encouraged to upgrade to either this release or the latest in our stable release line, which is currently X.Y.Z. Releases in the 1.2.z line are expected to stop in late spring 2019. Presuming no one wants to keep things going by the time a 1.2.12 VOTE passes, I'll modify it to say the branch is EOL. About a month later I'll do a dedicated EOL post to user@hbase, clean up project references to the release line, and then remove the release from dist.apache.org (it will remain along with all the others on archive.apache.org). [1]: https://s.apache.org/UEPy [2]: https://issues.apache.org/jira/browse/HBASE-22171
[jira] [Resolved] (HBASE-22176) Document how to make use of GitHub PR in HBase
[ https://issues.apache.org/jira/browse/HBASE-22176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-22176. --- Resolution: Duplicate Resolving as duplicate of HBASE-21701 > Document how to make use of GitHub PR in HBase > -- > > Key: HBASE-22176 > URL: https://issues.apache.org/jira/browse/HBASE-22176 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Duo Zhang >Priority: Major > > At least we need some guidelines for committers, so they do not click the > 'Merge Pull Request' button directly and mess up the commits, like me... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22196) Split TestRestartCluster
Duo Zhang created HBASE-22196:
-
Summary: Split TestRestartCluster
Key: HBASE-22196
URL: https://issues.apache.org/jira/browse/HBASE-22196
Project: HBase
Issue Type: Sub-task
Reporter: Duo Zhang
The logs for later tests are messed up with error messages, like
{noformat}
2019-04-09 09:41:11,717 WARN [LeaseRenewer:jenkins.hfs.12@localhost:41108]
hdfs.LeaseRenewer(468): Failed to renew lease for
[DFSClient_NONMAPREDUCE_400481390_21] for 55 seconds. Will retry shortly ...
java.net.ConnectException: Call From asf918.gq1.ygridcore.net/67.195.81.138 to
localhost:41108 failed on connection exception: java.net.ConnectException:
Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor79.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1480)
at org.apache.hadoop.ipc.Client.call(Client.java:1413)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy30.renewLease(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.renewLease(ClientNamenodeProtocolTranslatorPB.java:595)
at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy33.renewLease(Unknown Source)
at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:372)
at com.sun.proxy.$Proxy34.renewLease(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:901)
at org.apache.hadoop.hdfs.LeaseRenewer.renew(LeaseRenewer.java:423)
at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:448)
at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at
org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:615)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:713)
at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
at org.apache.hadoop.ipc.Client.call(Client.java:1452)
... 26 more
2019-04-09 09:41:11,949 WARN [RS_OPEN_REGION-regionserver/asf918:33671-1]
regionserver.HStore(1062): Failed flushing store file, retrying num=8
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:817)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2114)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
at
Re: hbase contributor
Granted. Enjoy your journey in hbase project and looking forward to your contribution (smile). Best Regards, Yu On Tue, 9 Apr 2019 at 13:19, 刘文文 wrote: > Hi, > I want to contribute to Apache HBase. Would you please give me the > contributor permission? My JIRA ID is wen.feiyi.
Re: 申请 JIRA Contributor 权限
Granted. Enjoy your journey in hbase project and looking forward to your contribution (smile). Best Regards, Yu On Tue, 9 Apr 2019 at 10:43, 郭康康 wrote: > Hi, > > I want to contribute to Apache HBase. > Would you please give me the contributor permission? > My JIRA ID is gk_coder. > >
[jira] [Created] (HBASE-22195) TestRestartCluster.testRetainAssignmentOnRestart is flakey
Duo Zhang created HBASE-22195:
-
Summary: TestRestartCluster.testRetainAssignmentOnRestart is flakey
Key: HBASE-22195
URL: https://issues.apache.org/jira/browse/HBASE-22195
Project: HBase
Issue Type: Bug
Reporter: Duo Zhang
{noformat}
java.lang.AssertionError: Values should be different. Actual: 1554802873115
at
org.apache.hadoop.hbase.master.TestRestartCluster.testRetainAssignmentOnRestart(TestRestartCluster.java:292)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-22153) Fix the flaky TestRestartCluster
[ https://issues.apache.org/jira/browse/HBASE-22153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-22153. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to branch-2.2+. > Fix the flaky TestRestartCluster > > > Key: HBASE-22153 > URL: https://issues.apache.org/jira/browse/HBASE-22153 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Duo Zhang >Priority: Major > > I guess it's related to HBASE-21565 or HBASE-21588 > Log can be see here: > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2902/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestRestartCluster-output.txt -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-22188) Make TestSplitMerge more stable
[ https://issues.apache.org/jira/browse/HBASE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-22188. --- Resolution: Fixed Assignee: Duo Zhang Hadoop Flags: Reviewed Fix Version/s: 2.1.5 2.3.0 2.2.0 3.0.0 Pushed to branch-2.1+. Thanks [~openinx] for reviewing. > Make TestSplitMerge more stable > --- > > Key: HBASE-22188 > URL: https://issues.apache.org/jira/browse/HBASE-22188 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.3.0, 2.1.5 > > > Sometimes it fails in pre commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22194) Snapshot unittests fail on Windows due to invalid file path uri
Bahram Chehrazy created HBASE-22194: --- Summary: Snapshot unittests fail on Windows due to invalid file path uri Key: HBASE-22194 URL: https://issues.apache.org/jira/browse/HBASE-22194 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 3.0.0, 2.2.0 Reporter: Bahram Chehrazy Assignee: Bahram Chehrazy While working on a patch for HBASE-22060, I noticed that a unit test started failing because region server crashed with NPE during initialization and after aborting the master. It turned out that access to the rssStub was not synchronized. The existing code in HRegionServer already takes care of this fact by copying and null checking in most places, but there are a couple ones that are not so careful. Those are in reportForDuty and abort methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
