[jira] [Resolved] (HBASE-26010) Backport HBASE-25703 and HBASE-26002 to branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-26010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-26010. -- Resolution: Won't Fix > Backport HBASE-25703 and HBASE-26002 to branch-2.3 > -- > > Key: HBASE-26010 > URL: https://issues.apache.org/jira/browse/HBASE-26010 > Project: HBase > Issue Type: Improvement > Components: backport > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 2.3.6 > > > Backport HBASE-25703 "Support conditional update in MultiRowMutationEndpoint" > and HBASE-26002 "MultiRowMutationEndpoint should return the result of the > conditional update" to branch-2.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26009) Backport HBASE-25766 "Introduce RegionSplitRestriction that restricts the pattern of the split point" to branch-2.3
[ https://issues.apache.org/jira/browse/HBASE-26009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-26009. -- Resolution: Won't Fix > Backport HBASE-25766 "Introduce RegionSplitRestriction that restricts the > pattern of the split point" to branch-2.3 > --- > > Key: HBASE-26009 > URL: https://issues.apache.org/jira/browse/HBASE-26009 > Project: HBase > Issue Type: Sub-task > Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 2.3.6 > > > Backport the parent issue to branch-2.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26010) Backport HBASE-25703 and HBASE-26002 to branch-2.3
Toshihiro Suzuki created HBASE-26010: Summary: Backport HBASE-25703 and HBASE-26002 to branch-2.3 Key: HBASE-26010 URL: https://issues.apache.org/jira/browse/HBASE-26010 Project: HBase Issue Type: Bug Components: backport Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Backport HBASE-25703 "Support conditional update in MultiRowMutationEndpoint" and HBASE-26002 "MultiRowMutationEndpoint should return the result of the conditional update" to branch-2.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26009) Backport HBASE-25766 "Introduce RegionSplitRestriction that restricts the pattern of the split point" to branch-2.3
Toshihiro Suzuki created HBASE-26009: Summary: Backport HBASE-25766 "Introduce RegionSplitRestriction that restricts the pattern of the split point" to branch-2.3 Key: HBASE-26009 URL: https://issues.apache.org/jira/browse/HBASE-26009 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Backport the parent issue to branch-2.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26002) MultiRowMutationEndpoint should return the result of the conditional update
[ https://issues.apache.org/jira/browse/HBASE-26002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-26002. -- Hadoop Flags: Reviewed Resolution: Fixed > MultiRowMutationEndpoint should return the result of the conditional update > --- > > Key: HBASE-26002 > URL: https://issues.apache.org/jira/browse/HBASE-26002 > Project: HBase > Issue Type: Improvement > Components: Coprocessors > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-2, 2.4.5 > > > HBASE-25703 added support for the conditional update in > MultiRowMutationEndpoint, but MultiRowMutationEndpoint doesn't return the > result of the conditional update (whether or not the mutations are executed). > In this Jira, we will make MultiRowMutationEndpoint return the result of the > conditional update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26002) MultiRowMutationEndpoint should return the result of the conditional update
Toshihiro Suzuki created HBASE-26002: Summary: MultiRowMutationEndpoint should return the result of the conditional update Key: HBASE-26002 URL: https://issues.apache.org/jira/browse/HBASE-26002 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki HBASE-25703 added support for the conditional update in MultiRowMutationEndpoint, but MultiRowMutationEndpoint doesn't return the result of the conditional update (whether or not the mutations are executed). In this Jira, we will make MultiRowMutationEndpoint return the result of the conditional update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] New HBase Committer Xiaolin Ha(哈晓琳)
Congratulations! Xiaolin Ha On Mon, May 17, 2021 at 2:55 PM Pankaj Kumar wrote: > Congratulations and welcome Xiaolin Ha.. > > Regards, > Pankaj > > On Sat, May 15, 2021, 7:41 PM 张铎(Duo Zhang) wrote: > > > On behalf of the Apache HBase PMC, I am pleased to announce that Xiaolin > > Ha(sunhelly) has accepted the PMC's invitation to become a committer on > the > > project. We appreciate all of Xiaolin's generous contributions thus far > and > > look forward to her continued involvement. > > > > Congratulations and welcome, Xiaolin Ha! > > > > 我很高兴代表Apache HBase PMC宣布哈晓琳已接受我们的邀请，成为Apache > > HBase项目的Committer。感谢哈晓琳一直以来为HBase项目做出的贡献，并期待她在未来继续承担更多的责任。 > > > > 欢迎哈晓琳！ > > >
[jira] [Resolved] (HBASE-25766) Introduce RegionSplitRestriction that restricts the pattern of the split point
[ https://issues.apache.org/jira/browse/HBASE-25766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25766. -- Hadoop Flags: Reviewed Resolution: Fixed > Introduce RegionSplitRestriction that restricts the pattern of the split point > -- > > Key: HBASE-25766 > URL: https://issues.apache.org/jira/browse/HBASE-25766 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 > > > As discussed in HBASE-25706, we can introduce RegionSplitRestriction that > restricts the pattern of the split point. > See the following comment for the details: > https://issues.apache.org/jira/browse/HBASE-25706?focusedCommentId=17310190=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17310190 > CC: [~zhangduo] [~stack] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] New HBase committer Geoffrey Jacoby
Congratulations Geoffrey! - Toshi > On Apr 11, 2021, at 0:12, Ankit Singhal wrote: > > Congratulations and welcome Geoffrey!! > > On Sat, Apr 10, 2021 at 5:48 PM Jan Hentschel < > jan.hentsc...@ultratendency.com> wrote: > >> Congratulations and welcome! >> >> From: Viraj Jasani >> Reply-To: "dev@hbase.apache.org" >> Date: Friday, April 9, 2021 at 1:24 PM >> To: hbase-dev , hbase-user >> Subject: [ANNOUNCE] New HBase committer Geoffrey Jacoby >> >> On behalf of the Apache HBase PMC I am pleased to announce that Geoffrey >> Jacoby has accepted the PMC's invitation to become a committer on the >> project. >> >> Thanks so much for the work you've been contributing. We look forward to >> your continued involvement. >> >> Congratulations and welcome, Geoffrey! >> >>
[jira] [Resolved] (HBASE-25706) Support specifying a base split policy class in KeyPrefixRegionSplitPolicy and DelimitedKeyPrefixRegionSplitPolicy
[ https://issues.apache.org/jira/browse/HBASE-25706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25706. -- Resolution: Won't Fix Closing this Jira and opened a new Jira HBASE-25766 to introduce RegionSplitPointRestriction. > Support specifying a base split policy class in KeyPrefixRegionSplitPolicy > and DelimitedKeyPrefixRegionSplitPolicy > -- > > Key: HBASE-25706 > URL: https://issues.apache.org/jira/browse/HBASE-25706 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > > Basically, I think we can use KeyPrefixRegionSplitPolicy and > DelimitedKeyPrefixRegionSplitPolicy along with other split policies. In this > Jira, we will support specifying a base split policy class in > KeyPrefixRegionSplitPolicy and DelimitedKeyPrefixRegionSplitPolicy to use > them with different split policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25766) Introduce RegionSplitPointRestriction that restricts the pattern of the split point
Toshihiro Suzuki created HBASE-25766: Summary: Introduce RegionSplitPointRestriction that restricts the pattern of the split point Key: HBASE-25766 URL: https://issues.apache.org/jira/browse/HBASE-25766 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.3 As discussed in HBASE-25706, we can introduce RegionSplitPointRestriction that restricts the pattern of the split point. See the following comment for the details: https://issues.apache.org/jira/browse/HBASE-25706?focusedCommentId=17310190=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17310190 CC: [~zhangduo] [~stack] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25703) Support conditional update in MultiRowMutationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-25703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25703. -- Hadoop Flags: Reviewed Resolution: Fixed > Support conditional update in MultiRowMutationEndpoint > -- > > Key: HBASE-25703 > URL: https://issues.apache.org/jira/browse/HBASE-25703 > Project: HBase > Issue Type: Improvement > Components: Coprocessors > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 > > > In some use cases, I think it will be useful that we can perform conditional > update in MultiRowMutationEndpoint. In this Jira, we can add support > conditional update in MultiRowMutationEndpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25706) Support specifying a base split policy class in KeyPrefixRegionSplitPolicy to make it more useful
Toshihiro Suzuki created HBASE-25706: Summary: Support specifying a base split policy class in KeyPrefixRegionSplitPolicy to make it more useful Key: HBASE-25706 URL: https://issues.apache.org/jira/browse/HBASE-25706 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Basically, I think we can use KeyPrefixRegionSplitPolicy along with other split policies. In this Jira, we will support specifying a base split policy class in KeyPrefixRegionSplitPolicy to use it with different split policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25702) Remove RowProcessor
[ https://issues.apache.org/jira/browse/HBASE-25702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25702. -- Hadoop Flags: Reviewed Resolution: Fixed > Remove RowProcessor > --- > > Key: HBASE-25702 > URL: https://issues.apache.org/jira/browse/HBASE-25702 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1 > > > As RowProcessor is deprecated, we can remove it in this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25704) Support conditional update in MultiRowMutationEndpoint
[ https://issues.apache.org/jira/browse/HBASE-25704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25704. -- Resolution: Duplicate > Support conditional update in MultiRowMutationEndpoint > -- > > Key: HBASE-25704 > URL: https://issues.apache.org/jira/browse/HBASE-25704 > Project: HBase > Issue Type: Improvement > Components: Coprocessors > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 > > > In some use cases, I think it will be useful that we can perform conditional > update in MultiRowMutationEndpoint. In this Jira, we can add support > conditional update in MultiRowMutationEndpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25704) Support conditional update in MultiRowMutationEndpoint
Toshihiro Suzuki created HBASE-25704: Summary: Support conditional update in MultiRowMutationEndpoint Key: HBASE-25704 URL: https://issues.apache.org/jira/browse/HBASE-25704 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 In some use cases, I think it will be useful that we can perform conditional update in MultiRowMutationEndpoint. In this Jira, we can add support conditional update in MultiRowMutationEndpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25703) Support conditional update in MultiRowMutationEndpoint
Toshihiro Suzuki created HBASE-25703: Summary: Support conditional update in MultiRowMutationEndpoint Key: HBASE-25703 URL: https://issues.apache.org/jira/browse/HBASE-25703 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 In some use cases, I think it will be useful that we can perform conditional update in MultiRowMutationEndpoint. In this Jira, we can add support conditional update in MultiRowMutationEndpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25686) [hbtop] Add some javadoc
[ https://issues.apache.org/jira/browse/HBASE-25686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25686. -- Hadoop Flags: Reviewed Resolution: Fixed > [hbtop] Add some javadoc > > > Key: HBASE-25686 > URL: https://issues.apache.org/jira/browse/HBASE-25686 > Project: HBase > Issue Type: Improvement > Components: hbtop > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25702) Remove RowProcessor
Toshihiro Suzuki created HBASE-25702: Summary: Remove RowProcessor Key: HBASE-25702 URL: https://issues.apache.org/jira/browse/HBASE-25702 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1 As RowProcessor is deprecated, we can remove it in this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25678) Support nonce operations for Increment/Append in RowMutations and CheckAndMutate
[ https://issues.apache.org/jira/browse/HBASE-25678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25678. -- Hadoop Flags: Reviewed Resolution: Fixed > Support nonce operations for Increment/Append in RowMutations and > CheckAndMutate > > > Key: HBASE-25678 > URL: https://issues.apache.org/jira/browse/HBASE-25678 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3 > > > Currently, we support nonce operations for single Increment/Append operations > but don't support for Increment/Append in RowMutations and CheckAndMutate. We > should support it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25686) [hbtop] Add some javadoc
Toshihiro Suzuki created HBASE-25686: Summary: [hbtop] Add some javadoc Key: HBASE-25686 URL: https://issues.apache.org/jira/browse/HBASE-25686 Project: HBase Issue Type: Improvement Components: hbtop Reporter: Toshihiro Suzuki -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25258) Backport HBASE-24776 "[hbtop] Support Batch mode" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-25258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25258. -- Fix Version/s: 1.7.0 Hadoop Flags: Reviewed Resolution: Fixed > Backport HBASE-24776 "[hbtop] Support Batch mode" to branch-1 > - > > Key: HBASE-25258 > URL: https://issues.apache.org/jira/browse/HBASE-25258 > Project: HBase > Issue Type: Sub-task >Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 1.7.0 > > > Backport parent issue to branch-1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25678) Support nonce operations for Increment/Append in RowMutations and CheckAndMutate
Toshihiro Suzuki created HBASE-25678: Summary: Support nonce operations for Increment/Append in RowMutations and CheckAndMutate Key: HBASE-25678 URL: https://issues.apache.org/jira/browse/HBASE-25678 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.3 Currently, we support nonce operations for single Increment/Append operations but don't support for Increment/Append in RowMutations and CheckAndMutate. We should support it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25575) Should validate Puts in RowMutations
[ https://issues.apache.org/jira/browse/HBASE-25575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25575. -- Hadoop Flags: Reviewed Resolution: Fixed > Should validate Puts in RowMutations > > > Key: HBASE-25575 > URL: https://issues.apache.org/jira/browse/HBASE-25575 > Project: HBase > Issue Type: Bug > Components: Client > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.2 > > > Currently, we don't validate the key value sizes of Puts in RowMutations. We > should do that. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25574) Revisit put/delete/increment/append related RegionObserver methods
[ https://issues.apache.org/jira/browse/HBASE-25574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25574. -- Hadoop Flags: Reviewed Resolution: Fixed > Revisit put/delete/increment/append related RegionObserver methods > -- > > Key: HBASE-25574 > URL: https://issues.apache.org/jira/browse/HBASE-25574 > Project: HBase > Issue Type: Improvement > Components: Coprocessors > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.2 > > > After HBASE-24602, increment/append operations are performed in > HRegion.batchMutate(). Accordingly, we should make some changes for the > increment/append related RegionObserver methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25576) Should not use HashMap (ConcurrentHashMap) or HashSet when using byte[] as a key of Map or an element of Set
Toshihiro Suzuki created HBASE-25576:
Summary: Should not use HashMap (ConcurrentHashMap) or HashSet
when using byte[] as a key of Map or an element of Set
Key: HBASE-25576
URL: https://issues.apache.org/jira/browse/HBASE-25576
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
I sometimes face the code using HashMap (ConcurrentHashMap) or HashSet when
using byte[] as a key of Map or an element of Set in the HBase code, which
could cause very confusing bugs.
We should use TreeMap (ConcurrentSkipListMap) or TreeSet
(ConcurrentSkipListSet) with Bytes.BYTES_COMPARATOR when using byte[] as a key
of Map or an element of Set as follows:
{code}
Map map1 = new TreeMap<>(Bytes.BYTES_COMPARATOR);
Map map2 = new ConcurrentSkipListMap<>(Bytes.BYTES_COMPARATOR);
Set set1 = new TreeSet<>(Bytes.BYTES_COMPARATOR);
Set set2 = new ConcurrentSkipListSet<>(Bytes.BYTES_COMPARATOR);
{code}
We should fix the existing ones in this Jira.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25575) Should validate Puts in RowMutations
Toshihiro Suzuki created HBASE-25575: Summary: Should validate Puts in RowMutations Key: HBASE-25575 URL: https://issues.apache.org/jira/browse/HBASE-25575 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, we don't validate Puts in RowMutations. We should do that. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25574) Revisit increment/append related RegionObserver methods
Toshihiro Suzuki created HBASE-25574: Summary: Revisit increment/append related RegionObserver methods Key: HBASE-25574 URL: https://issues.apache.org/jira/browse/HBASE-25574 Project: HBase Issue Type: Improvement Components: Coprocessors Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.2 After HBASE-24602, increment/append operations are performed in HRegion.batchMutate(). Accordingly, we should make some changes for the increment/append related RegionObserver methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] First release candidate for HBase 2.4.0 (RC1) is available
Thank you for raising the discussion. I thought changing the return type of the mutateRow method from void to Result in HBASE-25242 didn’t violate our release policy according to the doc some guys already mentioned. And looks like that was already agreed. Thanks, Toshihiro Suzuki > On Dec 5, 2020, at 4:19, Nick Dimiduk wrote: > > From the section "11.1. Aspirational Semantic Versioning" [0], under the > heading "Client Binary compatibility", we say: >> If a Client implements an HBase Interface, a recompile MAY be required > upgrading to a newer minor version (See release notes for warning about > incompatible changes). All effort will be made to provide a default > implementation so this case should not arise. > > This RC contains a change to client-facing API, not to an HBase interface. > However, I think this guidance holds for all means of consuming the client > API. Thus, my position is that this API change from HBASE-25242 is > acceptable under our terms of MINOR release compatibility, and that the > section of the book that I have cited should be updated, from "If a Client > implements an HBase Interface, ..." to "For any downstream code that > consumes an HBase IA.Public API, ..." > > Thanks, > Nick > > [0]: https://hbase.apache.org/book.html#hbase.versioning.post10 > > On Fri, Dec 4, 2020 at 9:15 AM Andrew Purtell > wrote: > >> This is a borderline change so I left it in but expected this discussion. >> Changing the return type of a method causes a binary incompatibility but >> not a source incompatibility. Making a compatibility method is difficult in >> this case because although the return type is considered part of the method >> signature the Java compiler only looks at parameters when deciding if a >> method duplicates another. So we can’t have the old method returning void >> and the new one returning Result in the same interface. The new method >> returning Result must also define additional parameters or accept a >> parameter of a different type. (At least, this is what I recall from >> earlier work, would be great if I’m wrong.) RowMutations is I would expect >> relatively uncommonly used. Finally, as an API improvement project this is >> important work. So I come down on the side of considering this an allowable >> change. That said, if the consensus is that it is not, it should be >> straightforward to change this method’s return type back to void and spin a >> new RC. >> >> >>> On Dec 3, 2020, at 11:04 PM, 张铎 wrote: >>> >>> I think for a minor release, we do not expect a drop-in replacement, so >>> changing the return value from void is fine? You do not need to change >> your >>> code, just need to recompile it. >>> >>> Thanks. >>> >>> Stack 于2020年12月4日周五 下午1:58写道： >>> >>>> +0 for now until my question below gets an answer. >>>> >>>> Signature is good, hash is good. CHANGES and RELEASENOTES look good too. >>>> Built from the src tgz. Artifact looks right when I untar it. >>>> Started it in standalone mode. >>>> UI looks good w/ nice 2.4.0 version on it (noticed new 'procedure time >>>> statistics'... nice). >>>> Loaded 10M rows w/ pe. Read them back out again. >>>> I've been running unit tests over the last few days. There are flakies >> that >>>> I've been trying >>>> to fix as I see them (HBASE-25353 is latest). I'll keep at it but don't >> see >>>> the last few as >>>> cause to hold up the RC (NIghtlies on branch-2 have been pretty good of >>>> late, see >>>> >>>> >> https://ci-hadoop.apache.org/view/HBase/job/HBase/job/HBase%20Nightly/job/branch-2/ >>>> ). >>>> >>>> Here is my question (Tosihrio?): >>>> >>>> Looking at API changes, this change looks problematic for a minor >> release: >>>> >>>> hbase-shaded-client-byo-hadoop-2.3.0.jar, Table.class >>>> package org.apache.hadoop.hbase.client >>>> [−] Table.mutateRow ( RowMutations rm ) *:* void 1 >>>> >>>> >> org/apache/hadoop/hbase/client/Table.mutateRow:(Lorg/apache/hadoop/hbase/client/RowMutations;)V >>>> >>>> ChangeEffect >>>> 1 Return value type has been changed from *void* to *Result*. This >> method >>>> has been removed because the return type is part of the method >> signature. A >>>> client program may be interrupted by *NoSuchMethodError* exception. >&
Re: [ANNOUNCE] New HBase committer Xin Sun
Congratulations Xin Sun! On Sat, Dec 5, 2020 at 10:38 AM Nick Dimiduk wrote: > Congratulations and thank you for your contributions! > > On Thu, Dec 3, 2020 at 01:13 Guanghao Zhang wrote: > > > Folks, > > > > On behalf of the Apache HBase PMC I am pleased to announce that Xin Sun > has > > accepted the PMC's invitation to become a committer on the project. > > > > We appreciate all of the great contributions Xin Sun has made to the > > community thus far and we look forward to his continued involvement. > > > > Allow me to be the first to congratulate Xin Sun on his new role! > > > > Thanks. > > >
Re: [ANNOUNCE] New HBase committer Yulin Niu
Congratulations Yulin! On Sat, Dec 5, 2020 at 10:38 AM Nick Dimiduk wrote: > Congratulations and thank you for your contributions! > > On Thu, Dec 3, 2020 at 01:11 Guanghao Zhang wrote: > > > Folks, > > > > On behalf of the Apache HBase PMC I am pleased to announce that Yulin Niu > > has accepted the PMC's invitation to become a committer on the project. > > > > We appreciate all of the great contributions Yulin has made to the > > community thus far and we look forward to his continued involvement. > > > > Allow me to be the first to congratulate Yulin on his new role! > > > > Thanks. > > >
[jira] [Created] (HBASE-25258) Backport HBASE-24776 "[hbtop] Support Batch mode" to branch-1
Toshihiro Suzuki created HBASE-25258: Summary: Backport HBASE-24776 "[hbtop] Support Batch mode" to branch-1 Key: HBASE-25258 URL: https://issues.apache.org/jira/browse/HBASE-25258 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Backport parent issue to branch-1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25242) Add Increment/Append support to RowMutations
Toshihiro Suzuki created HBASE-25242: Summary: Add Increment/Append support to RowMutations Key: HBASE-25242 URL: https://issues.apache.org/jira/browse/HBASE-25242 Project: HBase Issue Type: New Feature Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.0 Currently, RowMutations supports only Put and Delete. Supporting Increment/Append in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24210) Add Increment, Append and CheckAndMutate support to RowMutations
[ https://issues.apache.org/jira/browse/HBASE-24210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24210. -- Resolution: Won't Fix > Add Increment, Append and CheckAndMutate support to RowMutations > > > Key: HBASE-24210 > URL: https://issues.apache.org/jira/browse/HBASE-24210 > Project: HBase > Issue Type: New Feature > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, RowMutations supports only Put and Delete. Supporting Increment, > Append and CheckAndMutate in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24996) Support CheckAndMutate in Region.batchMutate()
[ https://issues.apache.org/jira/browse/HBASE-24996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24996. -- Resolution: Won't Fix > Support CheckAndMutate in Region.batchMutate() > -- > > Key: HBASE-24996 > URL: https://issues.apache.org/jira/browse/HBASE-24996 > Project: HBase > Issue Type: Sub-task > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > After HBASE-24602, Region.batchMutate() supports Put/Delete/Increment/Append > operations, but it doesn't support CheckAndMutate. If we support > CheckAndMutate Region.batchMutate(), we can perform > Put/Delete/Increment/Append/CheckAndMutate operations atomically, which is > very useful for some cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)
Toshihiro Suzuki created HBASE-25206:
Summary: Data loss can happen if a cloned table loses original
split region(delete table)
Key: HBASE-25206
URL: https://issues.apache.org/jira/browse/HBASE-25206
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki
Steps to reproduce are as follows:
1. Create a table and put some data into the table:
{code:java}
create 'test1','cf'
put 'test1','r1','cf','v1'
put 'test1','r2','cf','v2'
put 'test1','r3','cf','v3'
put 'test1','r4','cf','v4'
put 'test1','r5','cf','v5'
{code}
2. Take a snapshot for the table:
{code:java}
snapshot 'test1','snap_test'
{code}
3. Clone the snapshot to another table
{code:java}
clone_snapshot 'snap_test','test2'
{code}
4. Delete the snapshot
{code:java}
delete_snapshot 'snap_test'
{code}
5. Split the original table
{code:java}
split 'test1','r3'
{code}
6. Drop the original table
{code:java}
disable 'test1'
drop 'test1'
{code}
After that, we see the error like the following in RS log when opening the
regions of the cloned table:
{code:java}
2020-10-20 22:15:47,554 WARN [RS_OPEN_REGION-regionserver/10.0.1.8:0-0]
regionserver.HRegion(965): Failed initialize of region=
testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_-1603199739880,,1603199732706.92f431fab12aaded92a23513901daa5a.,
starting to roll back memstore
java.io.IOException: java.io.IOException: java.io.FileNotFoundException:
HFileLink
locations=[hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/.tmp/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/mobdir/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/archive/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402]
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1179)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1121)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1011)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:962)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7999)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7955)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7930)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7888)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7839)
at
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:132)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: java.io.FileNotFoundException: HFileLink
locations=[hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/.tmp/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/mobdir/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/archive/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705
[jira] [Resolved] (HBASE-25160) Refactor AccessController and VisibilityController
[ https://issues.apache.org/jira/browse/HBASE-25160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25160. -- Hadoop Flags: Reviewed Resolution: Fixed > Refactor AccessController and VisibilityController > -- > > Key: HBASE-25160 > URL: https://issues.apache.org/jira/browse/HBASE-25160 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > After HBASE-24602, the batchMutate() related coprocessor methods of > RegionObserver are called when increment/append operations are performed. We > can refactor AccessController and VisibilityController to make use of that. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25160) Refactor AccessController and VisibilityController
Toshihiro Suzuki created HBASE-25160: Summary: Refactor AccessController and VisibilityController Key: HBASE-25160 URL: https://issues.apache.org/jira/browse/HBASE-25160 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.0 After HBASE-24602, the batchMutate() related coprocessor methods of RegionObserver are called when increment/append operations are performed. We can refactor AccessController and VisibilityController to make use of that. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] New HBase Committer Zheng Wang(王正)
Congratulations! Zheng On Tue, Sep 29, 2020 at 2:43 AM Pankaj kr wrote: > Congratulations Zheng Wang and welcome...!! > > Regards, > Pankaj > -Original Message- > From: 张铎(Duo Zhang) [mailto:palomino...@gmail.com] > Sent: Thursday, September 24, 2020 7:54 AM > To: HBase Dev List ; hbase-user < > u...@hbase.apache.org>; user-zh > Subject: [ANNOUNCE] New HBase Committer Zheng Wang(王正) > > On behalf of the Apache HBase PMC, I am pleased to announce that Zheng > Wang has accepted the PMC's invitation to become a committer on the > project. We appreciate all of Zheng's generous contributions thus far and > look forward to his continued involvement. > > Congratulations and welcome, Zheng Wang! > > 我很高兴代表Apache HBase PMC宣布王正已接受我们的邀请，成为Apache > HBase项目的Committer。感谢王正一直以来为HBase项目做出的贡献，并期待他在未来继续承担更多的责任。 > > 欢迎王正！ >
[jira] [Resolved] (HBASE-25096) WAL size in RegionServer UI is wrong
[
https://issues.apache.org/jira/browse/HBASE-25096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-25096.
--
Hadoop Flags: Reviewed
Resolution: Fixed
> WAL size in RegionServer UI is wrong
>
>
> Key: HBASE-25096
> URL: https://issues.apache.org/jira/browse/HBASE-25096
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 2.2.7
>
>
> In MetricsRegionServerWrapperImpl, *walFileSize* is calculated by
> *provider.getLogFileSize()* twice as follows, which is causing the wrong WAL
> size in RegionServer UI:
> {code:java}
> walFileSize = (provider == null ? 0 : provider.getLogFileSize()) +
> (provider == null ? 0 : provider.getLogFileSize());
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25096) WAL size in RegionServer UI is wrong
Toshihiro Suzuki created HBASE-25096:
Summary: WAL size in RegionServer UI is wrong
Key: HBASE-25096
URL: https://issues.apache.org/jira/browse/HBASE-25096
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki
In MetricsRegionServerWrapperImpl, *walFileSize* is calculated by
*provider.getLogFileSize()* twice as follows, which is causing the wrong WAL
size in RegionServer UI:
{code:java}
walFileSize = (provider == null ? 0 : provider.getLogFileSize()) +
(provider == null ? 0 : provider.getLogFileSize());
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25008) Add document for "HBASE-24776 [hbtop] Support Batch mode"
[ https://issues.apache.org/jira/browse/HBASE-25008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-25008. -- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > Add document for "HBASE-24776 [hbtop] Support Batch mode" > - > > Key: HBASE-25008 > URL: https://issues.apache.org/jira/browse/HBASE-25008 > Project: HBase > Issue Type: Sub-task > Components: documentation, hbtop > Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Add a document for the batch mode of hbtop introduced in HBASE-24776 to the > ref guide. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23643) Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs"
[ https://issues.apache.org/jira/browse/HBASE-23643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-23643. -- Fix Version/s: 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed > Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client > drill downs" > - > > Key: HBASE-23643 > URL: https://issues.apache.org/jira/browse/HBASE-23643 > Project: HBase > Issue Type: Sub-task > Components: documentation, hbtop >Reporter: Toshihiro Suzuki >Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1 > > > Add the new feature of hbtop in HBASE-23065 to the ref guide. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24776) [hbtop] Support Batch mode
[ https://issues.apache.org/jira/browse/HBASE-24776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24776. -- Hadoop Flags: Reviewed Resolution: Fixed > [hbtop] Support Batch mode > -- > > Key: HBASE-24776 > URL: https://issues.apache.org/jira/browse/HBASE-24776 > Project: HBase > Issue Type: New Feature > Components: hbtop > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.2 > > > Similar to Unix's 'top' command, we can support Batch mode that allows us to > send hbtop command output to other programs or to a file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25008) Add document for "HBASE-24776 [hbtop] Support Batch mode"
Toshihiro Suzuki created HBASE-25008: Summary: Add document for "HBASE-24776 [hbtop] Support Batch mode" Key: HBASE-25008 URL: https://issues.apache.org/jira/browse/HBASE-25008 Project: HBase Issue Type: Sub-task Components: documentation, hbtop Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Add a document for the batch mode of hbtop introduced in HBASE-24776 to the ref guide. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24602) Add Increment and Append support to CheckAndMutate
[ https://issues.apache.org/jira/browse/HBASE-24602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24602. -- Resolution: Fixed > Add Increment and Append support to CheckAndMutate > -- > > Key: HBASE-24602 > URL: https://issues.apache.org/jira/browse/HBASE-24602 > Project: HBase > Issue Type: New Feature > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Currently, CheckAndMutate supports only Put and Delete. Supporting Increment > and Append would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24996) Support CheckAndMutate in Region.batchMutate()
Toshihiro Suzuki created HBASE-24996: Summary: Support CheckAndMutate in Region.batchMutate() Key: HBASE-24996 URL: https://issues.apache.org/jira/browse/HBASE-24996 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, Region.batchMutate() supports Put/Delete/Increment/Append operations, but it doesn't support CheckAndMutate. If we support CheckAndMutate Region.batchMutate(), we can perform Put/Delete/Increment/Append/CheckAndMutate operations atomically in a row, which is very useful for some cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24884) BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from command line parameters
[ https://issues.apache.org/jira/browse/HBASE-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24884. -- Hadoop Flags: Reviewed Resolution: Fixed > BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from > command line parameters > -- > > Key: HBASE-24884 > URL: https://issues.apache.org/jira/browse/HBASE-24884 > Project: HBase > Issue Type: Bug > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.2 > > > Currently, BulkLoadHFilesTool/LoadIncrementalHFiles doesn't accept -D options > from command line parameters. It should support them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24884) BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from command line parameters
Toshihiro Suzuki created HBASE-24884: Summary: BulkLoadHFilesTool/LoadIncrementalHFiles should accept -D options from command line parameters Key: HBASE-24884 URL: https://issues.apache.org/jira/browse/HBASE-24884 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, BulkLoadHFilesTool/LoadIncrementalHFiles doesn't accept -D options from command line parameters. It should support them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24680) Refactor the checkAndMutate code on the server side
[ https://issues.apache.org/jira/browse/HBASE-24680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24680. -- Hadoop Flags: Reviewed Resolution: Fixed > Refactor the checkAndMutate code on the server side > --- > > Key: HBASE-24680 > URL: https://issues.apache.org/jira/browse/HBASE-24680 > Project: HBase > Issue Type: Sub-task > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > Refactor the checkAndMutate code on the server side by using the > CheckAndMutate class (introduced in HBASE-8458) and the CheckAndMutateResult > class (introduced in HBASE-24650). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24830) Some tests involving RS crash fail with NullPointerException after HBASE-24632 in branch-2
Toshihiro Suzuki created HBASE-24830:
Summary: Some tests involving RS crash fail with
NullPointerException after HBASE-24632 in branch-2
Key: HBASE-24830
URL: https://issues.apache.org/jira/browse/HBASE-24830
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
In some tests involving RS crash in branch-2, the following
NullPointerException is happening repeatedly and the tests finally fail due to
timeout:
{code:java}
2020-08-06 16:03:43,101 ERROR [RS_LOG_REPLAY_OPS-regionserver/10.0.1.11:0-1]
handler.RSProcedureHandler(51): pid=17
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.SplitLogWorker.splitLog(SplitLogWorker.java:107)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:100)
at
org.apache.hadoop.hbase.regionserver.SplitWALCallable.call(SplitWALCallable.java:45)
at
org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:49)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24775) [hbtop] StoreFile size should be rounded off
[ https://issues.apache.org/jira/browse/HBASE-24775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24775. -- Hadoop Flags: Reviewed Resolution: Fixed > [hbtop] StoreFile size should be rounded off > > > Key: HBASE-24775 > URL: https://issues.apache.org/jira/browse/HBASE-24775 > Project: HBase > Issue Type: Bug > Components: hbtop > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.1, 1.7.0, 2.2.7 > > Attachments: Screen Shot.png, Screen Shot2.png > > > Currently, hbtop doesn't show StoreFile size rounded off, so when it's > reached 1GB it will be very ugly as follows: > !Screen Shot.png! > We should round off StoreFile size. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24776) [hbtop] Support Batch mode
Toshihiro Suzuki created HBASE-24776: Summary: [hbtop] Support Batch mode Key: HBASE-24776 URL: https://issues.apache.org/jira/browse/HBASE-24776 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Similar to Unix's 'top' command, we can support Batch mode that allows us to send top command output to other programs or to a file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24775) [hbtop] StoreFile size should be rounded off
Toshihiro Suzuki created HBASE-24775: Summary: [hbtop] StoreFile size should be rounded off Key: HBASE-24775 URL: https://issues.apache.org/jira/browse/HBASE-24775 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Attachments: Screen Shot.png Currently, hbtop doesn't show StoreFile size rounded off, so when it's reached 1GB it will be very ugly as follows: !Screen Shot.png! We should round off StoreFile size. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24650) Change the return types of the new checkAndMutate methods introduced in HBASE-8458
[
https://issues.apache.org/jira/browse/HBASE-24650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-24650.
--
Hadoop Flags: Incompatible change,Reviewed
Release Note:
This changed the return type of checkAndMutate methods to support
CheckAndMutate with Increment/Append.
The new APIs for the Table interface:
{code}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return A CheckAndMutateResult object that represents the result for the
CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default CheckAndMutateResult checkAndMutate(CheckAndMutate checkAndMutate)
throws IOException {
return checkAndMutate(Collections.singletonList(checkAndMutate)).get(0);
}
/**
* Batch version of checkAndMutate. The specified CheckAndMutates are batched
only in the sense
* that they are sent to a RS in one RPC, but each CheckAndMutate operation is
still executed
* atomically (and thus, each may fail independently of others).
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A list of CheckAndMutateResult objects that represents the result
for each
* CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default List checkAndMutate(List
checkAndMutates)
throws IOException {
throw new NotImplementedException("Add an implementation!");
}
{code}
The new APIs for the AsyncTable interface:
{code}
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return A {@link CompletableFuture}s that represent the result for the
CheckAndMutate.
*/
CompletableFuture checkAndMutate(CheckAndMutate
checkAndMutate);
/**
* Batch version of checkAndMutate. The specified CheckAndMutates are batched
only in the sense
* that they are sent to a RS in one RPC, but each CheckAndMutate operation is
still executed
* atomically (and thus, each may fail independently of others).
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A list of {@link CompletableFuture}s that represent the result for
each
* CheckAndMutate.
*/
List> checkAndMutate(
List checkAndMutates);
/**
* A simple version of batch checkAndMutate. It will fail if there are any
failures.
*
* @param checkAndMutates The list of rows to apply.
* @return A {@link CompletableFuture} that wrapper the result list.
*/
default CompletableFuture> checkAndMutateAll(
List checkAndMutates) {
return allOf(checkAndMutate(checkAndMutates));
}
{code}
Resolution: Fixed
> Change the return types of the new checkAndMutate methods introduced in
> HBASE-8458
> --
>
> Key: HBASE-24650
> URL: https://issues.apache.org/jira/browse/HBASE-24650
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> To support CheckAndMutate with Increment/Append, the new checkAndMutate
> methods introduced in HBASE-8458 need to return the result of the specified
> Increment/Append operation in addition to a boolean value represents whether
> it's successful or not. Currently, the methods return only boolean value(s),
> so we need to change the return types of the methods. The methods are
> unreleased yet currently, so I think it's no problem to change the return
> types of the methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24680) Refactor the checkAndMutate code on the server side
Toshihiro Suzuki created HBASE-24680: Summary: Refactor the checkAndMutate code on the server side Key: HBASE-24680 URL: https://issues.apache.org/jira/browse/HBASE-24680 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki Refactor the checkAndMutate code on the server side by using the CheckAndMutate class (introduced in HBASE-8458) and the CheckAndMutateResult class (introduced in HBASE-24650). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24650) Change the return types of the new CheckAndMutate methods introduced in HBASE-8458
Toshihiro Suzuki created HBASE-24650: Summary: Change the return types of the new CheckAndMutate methods introduced in HBASE-8458 Key: HBASE-24650 URL: https://issues.apache.org/jira/browse/HBASE-24650 Project: HBase Issue Type: Sub-task Components: Client Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0-alpha-1, 2.4.0 To support CheckAndMutate with Increment/Append, the new CheckAndMutate methods introduced in HBASE-8458 need to return the result of the specified Increment/Append operation in addition to a boolean value represents whether it's successful or not. Currently, the methods return only boolean value(s), so we need to change the return types of the methods. The methods are unreleased yet currently, so I think it's no problem to change the return types of the methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24600) Empty RegionAction added to MultiRequest in case of RowMutations/CheckAndMutate batch
[ https://issues.apache.org/jira/browse/HBASE-24600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24600. -- Fix Version/s: 2.2.6 2.1.10 2.4.0 2.3.1 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master, branch-2, branch-2.3, branch-2.2 and branch-2.1. Thank you for reviewing this [~zghao]! > Empty RegionAction added to MultiRequest in case of > RowMutations/CheckAndMutate batch > - > > Key: HBASE-24600 > URL: https://issues.apache.org/jira/browse/HBASE-24600 > Project: HBase > Issue Type: Bug > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.1.10, 2.2.6 > > > When a client sends RowMutations/CheckAndMutate batch requests, no Action > objects are added to the *builder* (RegionAction.Builder), so empty > RegionAction is added to MultiRequest at the following line: > https://github.com/apache/hbase/blob/3c319811799cb4c1f51fb5b43dd4743acd28052c/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java#L593 > We need to check if the *builder* has any Action objects or not here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24602) Add Increment and Append support to CheckAndMutate
Toshihiro Suzuki created HBASE-24602: Summary: Add Increment and Append support to CheckAndMutate Key: HBASE-24602 URL: https://issues.apache.org/jira/browse/HBASE-24602 Project: HBase Issue Type: New Feature Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, CheckAndMutate supports only Put, Delete and RowMutations. Supporting Increment and Append would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24600) Empty RegionAction added to MultiRequest in case of RowMutations/CheckAndMutate batch
Toshihiro Suzuki created HBASE-24600: Summary: Empty RegionAction added to MultiRequest in case of RowMutations/CheckAndMutate batch Key: HBASE-24600 URL: https://issues.apache.org/jira/browse/HBASE-24600 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki When a client sends RowMutations/CheckAndMutate batch requests, no Action objects are added to the *builder* (RegionAction.Builder), so empty RegionAction is added to MultiRequest at the following line: https://github.com/apache/hbase/blob/3c319811799cb4c1f51fb5b43dd4743acd28052c/hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java#L593 We need to check if the *builder* has any Action objects or not here. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-8458) Support for batch version of checkAndMutate()
[ https://issues.apache.org/jira/browse/HBASE-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-8458. - Resolution: Fixed > Support for batch version of checkAndMutate() > - > > Key: HBASE-8458 > URL: https://issues.apache.org/jira/browse/HBASE-8458 > Project: HBase > Issue Type: New Feature > Components: Client, regionserver >Reporter: Hari Mankude > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > The use case is that the user has multiple threads loading hundreds of keys > into a hbase table. Occasionally there are collisions in the keys being > uploaded by different threads. So for correctness, it is required to do > checkAndMutate() instead of a put(). However, doing a checkAndMutate() rpc > for every key update is non optimal. It would be good to have a batch version > of checkAndMutate() similar to batch put(). The client can partition the keys > on region boundaries. > The jira is NOT looking for any type of cross-row locking or multi-row > atomicity with checkAndMutate(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24529) hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles
[ https://issues.apache.org/jira/browse/HBASE-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24529. -- Hadoop Flags: Reviewed Resolution: Fixed > hbase.rs.evictblocksonclose is not honored when removing compacted files and > closing the storefiles > --- > > Key: HBASE-24529 > URL: https://issues.apache.org/jira/browse/HBASE-24529 > Project: HBase > Issue Type: Bug > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.1.10, 2.2.6 > > > Currently, when removing compacted files and closing the storefiles, RS > always does evict block caches for the store files. It should honor > hbase.rs.evictblocksonclose: > https://github.com/apache/hbase/blob/7b396e9b8ca93361de6a6c4bc8a40442db77c4da/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L2744 > https://github.com/apache/hbase/blob/7b396e9b8ca93361de6a6c4bc8a40442db77c4da/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L625 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24529) hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles
Toshihiro Suzuki created HBASE-24529: Summary: hbase.rs.evictblocksonclose is not honored when removing compacted files and closing the storefiles Key: HBASE-24529 URL: https://issues.apache.org/jira/browse/HBASE-24529 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, when removing compacted files and closing the storefiles, RS always does evict block caches for the store files. It should honor hbase.rs.evictblocksonclose: https://github.com/apache/hbase/blob/7b396e9b8ca93361de6a6c4bc8a40442db77c4da/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L2744 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24515) batch Increment/Append fails when retrying the RPC
[
https://issues.apache.org/jira/browse/HBASE-24515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-24515.
--
Hadoop Flags: Reviewed
Resolution: Fixed
> batch Increment/Append fails when retrying the RPC
> --
>
> Key: HBASE-24515
> URL: https://issues.apache.org/jira/browse/HBASE-24515
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.4.0, 2.1.10, 2.2.6
>
>
> When a client hits RPC timeout and sends a second RPC request for batch
> Increment/Append but the first RPC is already processed actually, the nonce
> of the RPC is saved in the RS.
> In this case, for the second RPC, the RS just reads the previous result and
> returns it to the client to avoid the Increment/Append is processed twice.
> At that time, for batch Increment/Append, we try to create a Get object from
> a CellScanner object in the following code:
>
> [https://github.com/apache/hbase/blob/66452afc09d8b82927e5e58565f97939faa22c7b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L773-L776]
> However, the CellScanner object is already consumed to create the
> Increment/Append object in the following code, and it fails with the
> following exception:
>
> [https://github.com/apache/hbase/blob/66452afc09d8b82927e5e58565f97939faa22c7b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L757]
> {code:java}
> 2020-06-06 14:09:06,153 WARN [hconnection-0x79c3903e-shared-pool3-t209]
> client.AsyncRequestFutureImpl: id=1, table=REF_Test, attempt=3/36,
> failureCount=1ops, last
> exception=org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException: Cell count of 1 but at index 0
> no cell returned: row: "xxx" mutate_type: INCREMENT timestamp:
> 9223372036854775807 durability: USE_DEFAULT time_range { from: 0 to:
> 9223372036854775807 } associated_cell_count: 1 nonce: 5281583076322914765
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toGet(ProtobufUtil.java:934)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:737)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:877)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2702)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42202)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24515) batch Increment/Append fails when retrying the RPC
Toshihiro Suzuki created HBASE-24515: Summary: batch Increment/Append fails when retrying the RPC Key: HBASE-24515 URL: https://issues.apache.org/jira/browse/HBASE-24515 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki When a client hits RPC timeout and sends a second RPC request for batch Increment/Append but the first RPC is already processed actually, the nonce of the RPC is saved in the RS. In this case, for the second RPC, the RS just reads the previous result and returns it to the client to avoid the Increment/Append is processed twice. At that time, for batch Increment/Append, we try to create a Get object from a CellScanner object in the following code: [https://github.com/apache/hbase/blob/66452afc09d8b82927e5e58565f97939faa22c7b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L773-L776] However, the CellScanner object is already consumed to create the Increment/Append object as follows, and it fails: [https://github.com/apache/hbase/blob/66452afc09d8b82927e5e58565f97939faa22c7b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L757] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review request, HBASE-8458 Support for batch version of checkAndMutate()
Could someone please take a look at this?
On Tue, May 5, 2020 at 8:03 AM Toshihiro Suzuki wrote:
> Hi folks!
>
> Could someone please review the following Jira and PR? Actually this
> feature is important for our use case.
>
> HBASE-8458 Support for batch version of checkAndMutate():
> https://issues.apache.org/jira/browse/HBASE-8458
>
> The RP:
> https://github.com/apache/hbase/pull/1648
>
> The highlights of the changes are as follows:
>
> - Introduced CheckAndMutate class that's used to perform CheckAndMutate
> operations. The following is the JavaDoc for this class:
> ```
> * Used to perform CheckAndMutate operations. Currently {@link Put}, {@link
> Delete}
> * and {@link RowMutations} are supported.
> *
> * This has a fluent style API to instantiate it, the code is like:
> *
> *
> * // A CheckAndMutate operation where do the specified action if the
> column (specified by the
> * // family and the qualifier) of the row equals to the specified value
> * CheckAndMutate checkAndMutate = new CheckAndMutate(row)
> * .ifEquals(family, qualifier, value)
> * .action(put);
> *
> * // A CheckAndMutate operation where do the specified action if the
> column (specified by the
> * // family and the qualifier) of the row doesn't exist
> * CheckAndMutate checkAndMutate = new CheckAndMutate(row)
> * .ifNotExists(family, qualifier)
> * .action(put);
> *
> * // A CheckAndMutate operation where do the specified action if the row
> matches the filter
> * CheckAndMutate checkAndMutate = new CheckAndMutate(row)
> * .ifMatches(filter)
> * .action(delete);
> *
> *
> ```
>
> - Added new checkAndMutate APIs to the Table and AsyncTable interfaces.
>
> The APIs for the Table interface:
> ```
> /**
> * checkAndMutate that atomically checks if a row matches the specified
> condition. If it does,
> * it performs the specified action.
> *
> * @param checkAndMutate The CheckAndMutate object.
> * @return boolean that represents the result for the CheckAndMutate.
> * @throws IOException if a remote or network exception occurs.
> */
> default boolean checkAndMutate(CheckAndMutate checkAndMutate) throws
> IOException {
>return checkAndMutate(Collections.singletonList(checkAndMutate))[0];
> }
>
> /**
> * Batch version of checkAndMutate.
> *
> * @param checkAndMutates The list of CheckAndMutate.
> * @return A array of boolean that represents the result for each
> CheckAndMutate.
> * @throws IOException if a remote or network exception occurs.
> */
> default boolean[] checkAndMutate(List checkAndMutates)
> throws IOException {
>throw new NotImplementedException("Add an implementation!");
> }
> ```
>
> The APIs for the AsyncTable interface:
> ```
> /**
> * checkAndMutate that atomically checks if a row matches the specified
> condition. If it does,
> * it performs the specified action.
> *
> * @param checkAndMutate The CheckAndMutate object.
> * @return A {@link CompletableFuture}s that represent the result for the
> CheckAndMutate.
> */
> CompletableFuture checkAndMutate(CheckAndMutate checkAndMutate);
>
> /**
> * Batch version of checkAndMutate.
> *
> * @param checkAndMutates The list of CheckAndMutate.
> * @return A list of {@link CompletableFuture}s that represent the result
> for each
> * CheckAndMutate.
> */
> List> checkAndMutate(List
> checkAndMutates);
>
> /**
> * A simple version of batch checkAndMutate. It will fail if there are any
> failures.
> *
> * @param checkAndMutates The list of rows to apply.
> * @return A {@link CompletableFuture} that wrapper the result boolean
> list.
> */
> default CompletableFuture> checkAndMutateAll(
> List checkAndMutates) {
> return allOf(checkAndMutate(checkAndMutates));
> }
> ```
>
> - Deprecated the old checkAndMutate APIs of the Table and AsyncTable
> interfaces.
>
> - This has Protocol Buffers level changes. I moved MultiRequest#condition
> to RegionAction and MultiResponse#processed to RegionActionResult. This is
> already discussed in the Jira:
>
> https://issues.apache.org/jira/browse/HBASE-8458?focusedCommentId=17064155=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17064155
>
> Thanks,
> Toshi
>
Re: [ANNOUNCE] New HBase committer Wei-Chiu Chuang
Congratulations Wei-Chiu! On Thu, May 14, 2020 at 8:25 PM Guangxu Cheng wrote: > Congratulations and welcome Wei-Chiu !!! > -- > Best Regards, > Guangxu > > > Reid Chan 于2020年5月14日周四 下午6:59写道： > > > > > Congratulations and welcome! > > > > > > -- > > > > Best regards, > > R.C > > > > > > > > > > From: ramkrishna vasudevan > > Sent: 14 May 2020 13:42 > > To: dev > > Subject: Re: [ANNOUNCE] New HBase committer Wei-Chiu Chuang > > > > Congratulations Wei-Chiu !!! > > > > Regards > > Ram > > > > On Thu, May 14, 2020 at 10:55 AM Viraj Jasani > wrote: > > > > > Congratulations Wei-Chiu !! > > > > > > On 2020/05/13 19:12:38, Sean Busbey wrote: > > > > Folks, > > > > > > > > On behalf of the Apache HBase PMC I am pleased to announce that > > Wei-Chiu > > > > Chuang has accepted the PMC's invitation to become a committer on the > > > > project. > > > > > > > > We appreciate all of the great contributions Wei-Chiu has made to the > > > > community thus far and we look forward to his continued involvement. > > > > > > > > Allow me to be the first to congratulate Wei-Chiu on his new role! > > > > > > > > thanks, > > > > busbey > > > > > > > > > >
Review request, HBASE-8458 Support for batch version of checkAndMutate()
Hi folks!
Could someone please review the following Jira and PR? Actually this
feature is important for our use case.
HBASE-8458 Support for batch version of checkAndMutate():
https://issues.apache.org/jira/browse/HBASE-8458
The RP:
https://github.com/apache/hbase/pull/1648
The highlights of the changes are as follows:
- Introduced CheckAndMutate class that's used to perform CheckAndMutate
operations. The following is the JavaDoc for this class:
```
* Used to perform CheckAndMutate operations. Currently {@link Put}, {@link
Delete}
* and {@link RowMutations} are supported.
*
* This has a fluent style API to instantiate it, the code is like:
*
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row equals to the specified value
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifEquals(family, qualifier, value)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the column
(specified by the
* // family and the qualifier) of the row doesn't exist
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifNotExists(family, qualifier)
* .action(put);
*
* // A CheckAndMutate operation where do the specified action if the row
matches the filter
* CheckAndMutate checkAndMutate = new CheckAndMutate(row)
* .ifMatches(filter)
* .action(delete);
*
*
```
- Added new checkAndMutate APIs to the Table and AsyncTable interfaces.
The APIs for the Table interface:
```
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return boolean that represents the result for the CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean checkAndMutate(CheckAndMutate checkAndMutate) throws
IOException {
return checkAndMutate(Collections.singletonList(checkAndMutate))[0];
}
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A array of boolean that represents the result for each
CheckAndMutate.
* @throws IOException if a remote or network exception occurs.
*/
default boolean[] checkAndMutate(List checkAndMutates)
throws IOException {
throw new NotImplementedException("Add an implementation!");
}
```
The APIs for the AsyncTable interface:
```
/**
* checkAndMutate that atomically checks if a row matches the specified
condition. If it does,
* it performs the specified action.
*
* @param checkAndMutate The CheckAndMutate object.
* @return A {@link CompletableFuture}s that represent the result for the
CheckAndMutate.
*/
CompletableFuture checkAndMutate(CheckAndMutate checkAndMutate);
/**
* Batch version of checkAndMutate.
*
* @param checkAndMutates The list of CheckAndMutate.
* @return A list of {@link CompletableFuture}s that represent the result
for each
* CheckAndMutate.
*/
List> checkAndMutate(List
checkAndMutates);
/**
* A simple version of batch checkAndMutate. It will fail if there are any
failures.
*
* @param checkAndMutates The list of rows to apply.
* @return A {@link CompletableFuture} that wrapper the result boolean list.
*/
default CompletableFuture> checkAndMutateAll(
List checkAndMutates) {
return allOf(checkAndMutate(checkAndMutates));
}
```
- Deprecated the old checkAndMutate APIs of the Table and AsyncTable
interfaces.
- This has Protocol Buffers level changes. I moved MultiRequest#condition
to RegionAction and MultiResponse#processed to RegionActionResult. This is
already discussed in the Jira:
https://issues.apache.org/jira/browse/HBASE-8458?focusedCommentId=17064155=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17064155
Thanks,
Toshi
[jira] [Created] (HBASE-24210) Add Increment, Append and CheckAndMutate support to RowMutations
Toshihiro Suzuki created HBASE-24210: Summary: Add Increment, Append and CheckAndMutate support to RowMutations Key: HBASE-24210 URL: https://issues.apache.org/jira/browse/HBASE-24210 Project: HBase Issue Type: New Feature Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Currently, RowMutations supports only Put and Delete. Supporting Increment, Append and CheckAndMutate in RowMutations would be helpful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24030) Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate()
[ https://issues.apache.org/jira/browse/HBASE-24030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-24030. -- Hadoop Flags: Reviewed Resolution: Fixed > Add necessary validations to HRegion.checkAndMutate() and > HRegion.checkAndRowMutate() > - > > Key: HBASE-24030 > URL: https://issues.apache.org/jira/browse/HBASE-24030 > Project: HBase > Issue Type: Bug > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Need to check that the following are fulfilled: > # The mutation type should be Put or Delete in HRegion.checkAndMutate(). > # The specified row should match the row of the Put/Delete/RowMutations in > HRegion.checkAndMutate() and HRegion.checkAndRowMutate(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24031) TestHRegion.testCheckAndMutate_WithFilters is flaky
[
https://issues.apache.org/jira/browse/HBASE-24031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-24031.
--
Resolution: Fixed
> TestHRegion.testCheckAndMutate_WithFilters is flaky
> ---
>
> Key: HBASE-24031
> URL: https://issues.apache.org/jira/browse/HBASE-24031
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Minor
> Fix For: 3.0.0
>
>
> {code}
> [ERROR] Tests run: 108, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 119.165 s <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestHRegion
> [ERROR]
> org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndMutate_WithFilters
> Time elapsed: 0.179 s <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndMutate_WithFilters(TestHRegion.java:2218)
> {code}
> This happens only in the master branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24031) TestHRegion.testCheckAndMutate_WithFilters is flaky
Toshihiro Suzuki created HBASE-24031:
Summary: TestHRegion.testCheckAndMutate_WithFilters is flaky
Key: HBASE-24031
URL: https://issues.apache.org/jira/browse/HBASE-24031
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki
Fix For: 3.0.0
{code}
[ERROR] Tests run: 108, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
119.165 s <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestHRegion
[ERROR]
org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndMutate_WithFilters
Time elapsed: 0.179 s <<< FAILURE!
java.lang.AssertionError: expected: but was:
at
org.apache.hadoop.hbase.regionserver.TestHRegion.testCheckAndMutate_WithFilters(TestHRegion.java:2218)
{code}
This happens only in the master branch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24030) Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate()
Toshihiro Suzuki created HBASE-24030: Summary: Add necessary validations to HRegion.checkAndMutate() and HRegion.checkAndRowMutate() Key: HBASE-24030 URL: https://issues.apache.org/jira/browse/HBASE-24030 Project: HBase Issue Type: Bug Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Fix For: 3.0.0, 2.3.0 Need to check that the following are fulfilled in HRegion.checkAndMutate() and HRegion.checkAndRowMutate(): # The mutation type should be Put or Delete. # The specified row should match the row of the Put/Delete/RowMutations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23925) Implement the checkAndMutate(row, filter) method of ThriftTable for Thrift
Toshihiro Suzuki created HBASE-23925: Summary: Implement the checkAndMutate(row, filter) method of ThriftTable for Thrift Key: HBASE-23925 URL: https://issues.apache.org/jira/browse/HBASE-23925 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23924) Implement the checkAndMutate(row, filter) method of RemoteHTable for Rest
Toshihiro Suzuki created HBASE-23924: Summary: Implement the checkAndMutate(row, filter) method of RemoteHTable for Rest Key: HBASE-23924 URL: https://issues.apache.org/jira/browse/HBASE-23924 Project: HBase Issue Type: Sub-task Reporter: Toshihiro Suzuki -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23146) Support CheckAndMutate with multiple conditions
[
https://issues.apache.org/jira/browse/HBASE-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-23146.
--
Release Note:
Add a checkAndMutate(row, filter) method in the AsyncTable interface and the
Table interface.
This method atomically checks if the row matches the specified filter. If it
does, it adds the Put/Delete/RowMutations.
This is a fluent style API, the code is like:
For Table interface:
{code}
table.checkAndMutate(row, filter).thenPut(put);
{code}
For AsyncTable interface:
{code}
table.checkAndMutate(row, filter).thenPut(put)
.thenAccept(succ -> {
if (succ) {
System.out.println("Check and put succeeded");
} else {
System.out.println("Check and put failed");
}
});
{code}
Assignee: Toshihiro Suzuki
Resolution: Fixed
> Support CheckAndMutate with multiple conditions
> ---
>
> Key: HBASE-23146
> URL: https://issues.apache.org/jira/browse/HBASE-23146
> Project: HBase
> Issue Type: New Feature
>Reporter: Toshihiro Suzuki
>Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, checkAndMutate supports only single condition. Supporting
> checkAndMutate with multi conditions is useful for some use cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23817) The message "Please make sure that backup is enabled on the cluster." is shown even when the backup feature is enabled
Toshihiro Suzuki created HBASE-23817:
Summary: The message "Please make sure that backup is enabled on
the cluster." is shown even when the backup feature is enabled
Key: HBASE-23817
URL: https://issues.apache.org/jira/browse/HBASE-23817
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
The following message is shown even when the backup feature is enabled, which
is confusing:
{code}
Please make sure that backup is enabled on the cluster. To enable backup, in
hbase-site.xml, set:
hbase.backup.enable=true
hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager
hbase.coprocessor.region.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.BackupObserver
and restart the cluster
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23165) [hbtop] Some modifications from HBASE-22988
[ https://issues.apache.org/jira/browse/HBASE-23165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-23165. -- Resolution: Fixed > [hbtop] Some modifications from HBASE-22988 > --- > > Key: HBASE-23165 > URL: https://issues.apache.org/jira/browse/HBASE-23165 > Project: HBase > Issue Type: Improvement > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9 > > > Some modifications happened in the review of HBASE-22988. We can forward port > them to the master branch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23643) Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs"
Toshihiro Suzuki created HBASE-23643: Summary: Add document for "HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs" Key: HBASE-23643 URL: https://issues.apache.org/jira/browse/HBASE-23643 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Add the new feature of hbtop in HBASE-23065 to the ref guide. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [ANNOUNCE] New HBase committer Viraj Jasani
Congratulations! Viraj On Sat, Dec 28, 2019 at 3:13 AM Andrew Purtell wrote: > Congratulations and welcome Viraj, thanks for all of your efforts so far. > > > On Dec 27, 2019, at 5:02 AM, Peter Somogyi wrote: > > > > On behalf of the Apache HBase PMC I am pleased to announce that > > Viraj Jasani has accepted the PMC's invitation to become a > > commiter on the project. > > > > Thanks so much for the work you've been contributing. We look forward > > to your continued involvement. > > > > Congratulations and welcome! >
[jira] [Created] (HBASE-23581) Creating table gets stuck when specifying an invalid split policy as METADATA
Toshihiro Suzuki created HBASE-23581:
Summary: Creating table gets stuck when specifying an invalid
split policy as METADATA
Key: HBASE-23581
URL: https://issues.apache.org/jira/browse/HBASE-23581
Project: HBase
Issue Type: Bug
Environment: HDP-3.1.0
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki
We can reproduce this issue as follows:
{code}
create 'test', "f", {METADATA => {'hbase.regionserver.region.split.policy' =>
'UNDEFINED'}}
{code}
After running this, creating table will get stuck. And it looks like this is
because opening region fails with ClassNotFoundException:
{code}
2019-12-16 06:45:03,671 ERROR [RS_OPEN_REGION-regionserver/c126-node2:16020-17]
handler.OpenRegionHandler: Failed open of
region=test,,1576477039045.7435965ddb2229c62d926b3ee963dcf3.
java.io.IOException: Unable to load configured region split policy 'UNDEFINED'
for table 'test'
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPolicyClass(RegionSplitPolicy.java:132)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkClassLoading(HRegion.java:7162)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7083)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7043)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7015)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6973)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6924)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: UNDEFINED
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPolicyClass(RegionSplitPolicy.java:127)
... 12 more
{code}
We should have sanity checks for the properties specified in METADATA.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-22096) /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file
[ https://issues.apache.org/jira/browse/HBASE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-22096. -- Resolution: Fixed > /storeFile.jsp shows CorruptHFileException when the storeFile is a reference > file > - > > Key: HBASE-22096 > URL: https://issues.apache.org/jira/browse/HBASE-22096 > Project: HBase > Issue Type: Bug > Components: UI > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3, 2.1.9, 1.4.13 > > Attachments: HBASE-22096.branch-2.2.addendum.v1.patch, screanshot.png > > > When the storeFile is a reference file, /storeFile.jsp for the storeFile > shows the following error: > !screanshot.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-22096) /storeFile.jsp shows CorruptHFileException when the storeFile is a reference file
[ https://issues.apache.org/jira/browse/HBASE-22096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki reopened HBASE-22096: -- > /storeFile.jsp shows CorruptHFileException when the storeFile is a reference > file > - > > Key: HBASE-22096 > URL: https://issues.apache.org/jira/browse/HBASE-22096 > Project: HBase > Issue Type: Bug > Components: UI > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3, 2.1.9 > > Attachments: HBASE-22096.branch-2.2.addendum.v1.patch, screanshot.png > > > When the storeFile is a reference file, /storeFile.jsp for the storeFile > shows the following error: > !screanshot.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23359) RS going down with NPE when splitting a region with compaction disabled in branch-1
[
https://issues.apache.org/jira/browse/HBASE-23359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-23359.
--
Resolution: Fixed
> RS going down with NPE when splitting a region with compaction disabled in
> branch-1
> ---
>
> Key: HBASE-23359
> URL: https://issues.apache.org/jira/browse/HBASE-23359
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 1.6.0
>
>
> Trying to backport HBASE-22096 to brach-1, I faced the issue where a RS goes
> down with NPE when splitting a region with compaction disabled.
> The steps to reproduce this issue are as follows:
> {code}
> compaction_switch false
> create "test", "cf"
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val#{i}"}
> split "test"
> {code}
> Looking at the regionserver log, I saw the following log:
> {code}
> 2019-12-03 22:25:38,611 INFO [RS:0;10.0.1.11:53504-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> test,,1575379535506.50e322ec68162025e17cddffdc2fb17e.; null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1834)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$Rejection.rejectedExecution(CompactSplitThread.java:656)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:401)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestSystemCompaction(CompactSplitThread.java:348)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2111)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2097)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.openDaughters(SplitTransactionImpl.java:478)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsAfterPONR(SplitTransactionImpl.java:549)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:532)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:153)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-12-03 22:25:38,613 FATAL [RS:0;10.0.1.11:53504-splits-0]
> regionserver.HRegionServer: ABORTING region server
> 10.0.1.11,53504,1575379011279: Abort; we got an error after point-of-no-return
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-23359) RS going down with NPE when splitting a region with compaction disabled in branch-1
[
https://issues.apache.org/jira/browse/HBASE-23359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki reopened HBASE-23359:
--
Reopening. Will push it to branch-1.4.
> RS going down with NPE when splitting a region with compaction disabled in
> branch-1
> ---
>
> Key: HBASE-23359
> URL: https://issues.apache.org/jira/browse/HBASE-23359
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 1.6.0
>
>
> Trying to backport HBASE-22096 to brach-1, I faced the issue where a RS goes
> down with NPE when splitting a region with compaction disabled.
> The steps to reproduce this issue are as follows:
> {code}
> compaction_switch false
> create "test", "cf"
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val#{i}"}
> split "test"
> {code}
> Looking at the regionserver log, I saw the following log:
> {code}
> 2019-12-03 22:25:38,611 INFO [RS:0;10.0.1.11:53504-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> test,,1575379535506.50e322ec68162025e17cddffdc2fb17e.; null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1834)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$Rejection.rejectedExecution(CompactSplitThread.java:656)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:401)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestSystemCompaction(CompactSplitThread.java:348)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2111)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2097)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.openDaughters(SplitTransactionImpl.java:478)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsAfterPONR(SplitTransactionImpl.java:549)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:532)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:153)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-12-03 22:25:38,613 FATAL [RS:0;10.0.1.11:53504-splits-0]
> regionserver.HRegionServer: ABORTING region server
> 10.0.1.11,53504,1575379011279: Abort; we got an error after point-of-no-return
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23303) Add security headers to REST server/info page
[
https://issues.apache.org/jira/browse/HBASE-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-23303.
--
Resolution: Fixed
> Add security headers to REST server/info page
> -
>
> Key: HBASE-23303
> URL: https://issues.apache.org/jira/browse/HBASE-23303
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Affects Versions: 3.0.0, 2.0.6, 2.1.7, 2.2.2
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Vulnerability scanners suggest that the following extra headers should be
> added to both Info/Rest server endpoints which are exposed by {{hbase-rest}}
> project.
> * X-Frame-Options: SAMEORIGIN
> * X-Xss-Protection: 1; mode=block
> * X-Content-Type-Options: nosniff
> * Strict-Transport-Security: “max-age=63072000;includeSubDomains;preload”
> * Content-Security-Policy: default-src https: data: 'unsafe-inline'
> 'unsafe-eval'
> Info server already has "X-Frame-Options: DENY" which is more restrictive
> than "SAMEORIGIN", so it's probably fine. All of three headers are missing
> from REST responses.
> I'll put together a patch to resolve this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23359) RS going down with NPE when splitting a region with compaction disabled in branch-1
[
https://issues.apache.org/jira/browse/HBASE-23359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Toshihiro Suzuki resolved HBASE-23359.
--
Assignee: Toshihiro Suzuki
Resolution: Fixed
> RS going down with NPE when splitting a region with compaction disabled in
> branch-1
> ---
>
> Key: HBASE-23359
> URL: https://issues.apache.org/jira/browse/HBASE-23359
> Project: HBase
> Issue Type: Bug
> Reporter: Toshihiro Suzuki
> Assignee: Toshihiro Suzuki
>Priority: Major
> Fix For: 1.6.0
>
>
> Trying to backport HBASE-22096 to brach-1, I faced the issue where a RS goes
> down with NPE when splitting a region with compaction disabled.
> The steps to reproduce this issue are as follows:
> {code}
> compaction_switch false
> create "test", "cf"
> (0...2000).each{|i| put "test", "row#{i}", "cf:col", "val#{i}"}
> split "test"
> {code}
> Looking at the regionserver log, I saw the following log:
> {code}
> 2019-12-03 22:25:38,611 INFO [RS:0;10.0.1.11:53504-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> test,,1575379535506.50e322ec68162025e17cddffdc2fb17e.; null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1834)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$Rejection.rejectedExecution(CompactSplitThread.java:656)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:401)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestSystemCompaction(CompactSplitThread.java:348)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2111)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2097)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.openDaughters(SplitTransactionImpl.java:478)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsAfterPONR(SplitTransactionImpl.java:549)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:532)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:153)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-12-03 22:25:38,613 FATAL [RS:0;10.0.1.11:53504-splits-0]
> regionserver.HRegionServer: ABORTING region server
> 10.0.1.11,53504,1575379011279: Abort; we got an error after point-of-no-return
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23359) RS going down with NPE when splitting a region with compaction disabled in branch-1
Toshihiro Suzuki created HBASE-23359:
Summary: RS going down with NPE when splitting a region with
compaction disabled in branch-1
Key: HBASE-23359
URL: https://issues.apache.org/jira/browse/HBASE-23359
Project: HBase
Issue Type: Bug
Reporter: Toshihiro Suzuki
Trying to backport HBASE-22096 to brach-1, I faced the issue where a RS goes
down with NPE when splitting a region with compaction disabled.
The steps to reproduce this issue are as follows:
{code}
compaction_switch false
create "test", "cf"
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val#{i}"}
split "test"
{code}
Looking at the regionserver log, I saw the following log:
{code}
2019-12-03 22:25:38,611 INFO [RS:0;10.0.1.11:53504-splits-0]
regionserver.SplitRequest: Running rollback/cleanup of failed split of
test,,1575379535506.50e322ec68162025e17cddffdc2fb17e.; null
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1834)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$Rejection.rejectedExecution(CompactSplitThread.java:656)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:401)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestSystemCompaction(CompactSplitThread.java:348)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2111)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2097)
at
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.openDaughters(SplitTransactionImpl.java:478)
at
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsAfterPONR(SplitTransactionImpl.java:549)
at
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:532)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:153)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-12-03 22:25:38,613 FATAL [RS:0;10.0.1.11:53504-splits-0]
regionserver.HRegionServer: ABORTING region server
10.0.1.11,53504,1575379011279: Abort; we got an error after point-of-no-return
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [ANNOUNCE] New HBase committer Ankit Singhal
Congratulations! Ankit On Wed, Nov 13, 2019 at 6:56 AM Esteban Gutierrez wrote: > Congrats, Ankit! > -- > Cloudera, Inc. > > > > On Tue, Nov 12, 2019 at 2:59 PM Jan Hentschel < > jan.hentsc...@ultratendency.com> wrote: > > > Congratulations and welcome! > > > > From: Josh Elser > > Reply-To: "dev@hbase.apache.org" > > Date: Tuesday, November 12, 2019 at 5:39 PM > > To: dev > > Subject: [ANNOUNCE] New HBase committer Ankit Singhal > > > > On behalf of the Apache HBase PMC, I'm pleased to announce that Ankit > > Singhal has accepted our invitation to become an HBase committer. > > > > Thanks for all of your contributions to the HBase project and we look > > forward to your continued growth and participation. > > > > Congratulations! > > > > >
Re: [ANNOUNCE] Please welcome Balazs Meszaros to the Apache HBase PMC
Congratulations! Balazs On Sat, Nov 2, 2019 at 9:16 PM Pankaj kr wrote: > > Congratulations Balazs Meszaros..!! > > Regards, > Pankaj > > -Original Message- > From: Sean Busbey [mailto:bus...@apache.org] > Sent: 24 October 2019 20:05 > To: dev ; Hbase-User > Subject: [ANNOUNCE] Please welcome Balazs Meszaros to the Apache HBase PMC > > On behalf of the Apache HBase PMC I am pleased to announce that Balazs > Meszaros has accepted our invitation to become a PMC member on the HBase > project. We appreciate Balazs stepping up to take more responsibility in > the HBase project. > > Please join me in welcoming Balazs to the HBase PMC! > > > > As a reminder, if anyone would like to nominate another person as a > committer or PMC member, even if you are not currently a committer or PMC > member, you can always drop a note to priv...@hbase.apache.org to let us > know. >
Re: [ANNOUNCE] Please welcome Wellington Chevreuil to the Apache HBase PMC
Congratulations! Wellington On Sat, Nov 2, 2019 at 9:14 PM Pankaj kr wrote: > Congratulations Wellington..!! > > Regards, > Pankaj > > > -Original Message- > From: Sean Busbey [mailto:bus...@apache.org] > Sent: 24 October 2019 01:47 > To: dev ; Hbase-User > Subject: [ANNOUNCE] Please welcome Wellington Chevreuil to the Apache > HBase PMC > > On behalf of the Apache HBase PMC I am pleased to announce that Wellington > Chevreuil has accepted our invitation to become a PMC member on the HBase > project. We appreciate Wellington stepping up to take more responsibility > in the HBase project. > > Please join me in welcoming Wellington to the HBase PMC! > > > > As a reminder, if anyone would like to nominate another person as a > committer or PMC member, even if you are not currently a committer or PMC > member, you can always drop a note to priv...@hbase.apache.org to let us > know. >
Re: [ANNOUNCE] Please welcome Sakthi to the Apache HBase PMC
Congratulations! Sakthi On Sat, Nov 2, 2019 at 9:13 PM Pankaj kr wrote: > Congratulations Sakthi..!! > > Regards, > Pankaj > > > -Original Message- > From: Sean Busbey [mailto:bus...@apache.org] > Sent: 24 October 2019 01:45 > To: dev ; Hbase-User > Subject: [ANNOUNCE] Please welcome Sakthi to the Apache HBase PMC > > On behalf of the Apache HBase PMC I am pleased to announce that Sakthi has > accepted our invitation to become a PMC member on the HBase project. We > appreciate Sakthi stepping up to take more responsibility in the HBase > project. > > Please join me in welcoming Jan to the HBase PMC! > > > > As a reminder, if anyone would like to nominate another person as a > committer or PMC member, even if you are not currently a committer or PMC > member, you can always drop a note to priv...@hbase.apache.org to let us > know. >
[jira] [Created] (HBASE-23166) [hbtop] Use a terminal library instead of writing our own
Toshihiro Suzuki created HBASE-23166: Summary: [hbtop] Use a terminal library instead of writing our own Key: HBASE-23166 URL: https://issues.apache.org/jira/browse/HBASE-23166 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Currently, we use a custom terminal implementation in hbtop. Ideally, we should use a 3rd party terminal/libs instead of writing our own. This is discussed in the following: https://github.com/apache/hbase/pull/476 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23165) [hbtop] Some modifications from HBASE-22988
Toshihiro Suzuki created HBASE-23165: Summary: [hbtop] Some modifications from HBASE-22988 Key: HBASE-23165 URL: https://issues.apache.org/jira/browse/HBASE-23165 Project: HBase Issue Type: Improvement Reporter: Toshihiro Suzuki Assignee: Toshihiro Suzuki Some modifications happened in the review of HBASE-22988. We can forward port them to the master branch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23115) Unit change for StoreFileSize and MemStoreSize
[ https://issues.apache.org/jira/browse/HBASE-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-23115. -- Resolution: Fixed > Unit change for StoreFileSize and MemStoreSize > -- > > Key: HBASE-23115 > URL: https://issues.apache.org/jira/browse/HBASE-23115 > Project: HBase > Issue Type: Bug > Components: metrics, UI >Affects Versions: 3.0.0 >Reporter: Karthik Palanisamy >Assignee: Karthik Palanisamy >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.2, 2.1.8 > > Attachments: Units.pdf > > > StoreFileSize and MemstoreSize usually returned in MBs (link1, link2) but > few jsp pages contain inaccurate unit. The reason is table.jsp (link3) use > org.apache.hadoop.util.StringUtils.byteDesc(long len), this will perform > longtostring conversion and returns its unit(B, KB, MB, GB, TB, PB) based on > length. The concern here (link4) is computation (ByteVal/1024/1024) will > output always lesser than 1 for store contains few bytes or few kbs. Also, > typecast will not round up to its nearest value. > I think the best option is changing unit in table.jsp instead of changing > code, otherwise we may end up doing many refactors from getMemStoreSizeMB, > setMemStoreSizeMB, hasMemStoreSizeMB, getStorefileSizeMB, > setStorefileSizeMB,.. > > Please find the attachment, a simple example is posted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23146) Support CheckAnd* with multi conditions
Toshihiro Suzuki created HBASE-23146: Summary: Support CheckAnd* with multi conditions Key: HBASE-23146 URL: https://issues.apache.org/jira/browse/HBASE-23146 Project: HBase Issue Type: New Feature Reporter: Toshihiro Suzuki Currently, checkAnd* supports only single condition. Supporting checkAnd* with multi conditions is useful for some use cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-22992) Blog post for hbtop on hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-22992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-22992. -- Resolution: Fixed > Blog post for hbtop on hbase.apache.org > --- > > Key: HBASE-22992 > URL: https://issues.apache.org/jira/browse/HBASE-22992 > Project: HBase > Issue Type: Sub-task > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-22986) Add documentation for hbtop
[ https://issues.apache.org/jira/browse/HBASE-22986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki resolved HBASE-22986. -- Resolution: Fixed > Add documentation for hbtop > --- > > Key: HBASE-22986 > URL: https://issues.apache.org/jira/browse/HBASE-22986 > Project: HBase > Issue Type: Sub-task > Components: documentation, hbtop > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Minor > Fix For: 3.0.0 > > > We already have README for hbtop, so we can make the refguide refer to this: > [https://github.com/apache/hbase/blob/master/hbase-hbtop/README.md] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-22992) Blog post for hbtop on hbase.apache.org
[ https://issues.apache.org/jira/browse/HBASE-22992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiro Suzuki reopened HBASE-22992: -- > Blog post for hbtop on hbase.apache.org > --- > > Key: HBASE-22992 > URL: https://issues.apache.org/jira/browse/HBASE-22992 > Project: HBase > Issue Type: Sub-task > Reporter: Toshihiro Suzuki > Assignee: Toshihiro Suzuki >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23064) [hbtop] Rewrite the parsing logic to a better way in RecordFilter
Toshihiro Suzuki created HBASE-23064: Summary: [hbtop] Rewrite the parsing logic to a better way in RecordFilter Key: HBASE-23064 URL: https://issues.apache.org/jira/browse/HBASE-23064 Project: HBase Issue Type: Improvement Components: hbtop Reporter: Toshihiro Suzuki As discussed in the following, we can rewrite the parsing logic to a better way (like index-substring way or pattern-matching way?) in RecordFilter: https://github.com/apache/hbase/pull/647#discussion_r326979376 -- This message was sent by Atlassian Jira (v8.3.4#803005)
