[jira] [Created] (HBASE-21750) Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized
Zheng Hu created HBASE-21750: Summary: Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized Key: HBASE-21750 URL: https://issues.apache.org/jira/browse/HBASE-21750 Project: HBase Issue Type: Sub-task Reporter: Zheng Hu Assignee: Zheng Hu Fix For: 3.0.0, 2.2.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21749) RS UI may throw NPE and make rs-status page inaccessible with multiwal and replication
Nihal Jain created HBASE-21749:
--
Summary: RS UI may throw NPE and make rs-status page inaccessible
with multiwal and replication
Key: HBASE-21749
URL: https://issues.apache.org/jira/browse/HBASE-21749
Project: HBase
Issue Type: Bug
Components: Replication, UI
Reporter: Nihal Jain
Assignee: Nihal Jain
Sometimes RS UI is unable to open as we get a NPE; This happens because
{{shipper.getCurrentPath()}} may return null.
We should have a null check @
[ReplicationSource.java#L331|https://github.com/apache/hbase/blob/a2f6768acdc30b789c7cb8482b9f4352803f60a1/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java#L331]
{code:java}
Path currentPath = shipper.getCurrentPath();
try {
fileSize = getFileSize(currentPath);
} catch (IOException e) {
LOG.warn("Ignore the exception as the file size of HLog only affects
the web ui", e);
fileSize = -1;
}{code}
!0b8e95c7-6715-42bf-88d2-b2edc9215022.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: About how features are integrated to different HBase versions
I can speak for myself and assume others backporting to branch-1 have similar
motivations. We are backporting interesting and useful and reasonable
improvements, because branch-1 is still useful and in production, and will be
for the foreseeable future. Changes like perf improvements and
operability/tooling/UI improvements should not be denied to branch-1 users if
some contributors and committers are putting in the effort to maintain it. I
agree major new features should not be backported. I doubt anyone will try to
backport AMv2 or IMC. I won’t, for sure. Not interested in destabilizing
change.
Personally I work for an organization that runs branch-1 derived code in
production. You can stop my maintenance activities in the community but that
would just mean I make a fork and take the work internal. Why not let me
contribute these efforts back to the community? It is your choice. I don't see
why we can't have activity both in branch-1 and branch-2. To each their own.
Let the community organically decide what is right by do-ocracy.
If you ask me why aren’t we using branch-2 now, I have to say unfortunately
it’s not production ready for us. Refer to the recent discussion about hbck2
for one concern. Another is a colleague did a trivial test with head of
branch-2 and measured a 50% perf degradation in scan performance. Now, that is
just a one off test. But it causes concern. We are swamped already with run the
business concerns. We can’t take on the additional risk at this time. Perhaps
in the future we will have more bandwidth to contribute to branch-2 efforts.
> On Jan 20, 2019, at 9:29 PM, Allan Yang wrote:
>
> {quote}
> For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
> but 2.1.1 should have all features which 1.5.0 has.
> {quote}
> I don't think can work, normal user mostly won't care about the release
> time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher version
> includes all the feature in lower version.
>
> I don't get the point why we are now still backporting new features from
> branch-2 to branch-1. Yes, there is many 1.x cluster in production, so we
> need to release 1.x versions to fix bugs and keep them stable, as the
> stable point is still in 1.x.
> And at the same time, we should try to move on to 2.x, making branch-1 as a
> bugfix branch for sometime before deprecating it. As far as I see, branch-1
> is still very 'active', too active I think.
> If we stop backport features from branch-2 to branch-1, then there is no
> problem, IMHO.
> Best Regards
> Allan Yang
>
>
> Andrew Purtell 于2019年1月20日周日 上午5:27写道：
>
>> As branch RM for branch-1 I will always check to make sure a commit there
>> has first been committed to branch-2. There will always be an upgrade path
>> from a branch-1 based release to a branch-2 based release. The relevant
>> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA will
>> be linked to the one for the branch-2 commit. When making the release notes
>> we will be looking at these things (or should, anyway). We can update the
>> upgrade paths documentation whenever we find this kind of linkage. Perhaps
>> we can describe this for future RMs in the how to release section of the
>> doc? Does this satisfy the concerns?
>>
>>> On Jan 18, 2019, at 11:47 PM, Sean Busbey wrote:
>>>
>>> I agree with Andrew that we can't both have maintenance releases and
>> expect
>>> every feature in ongoing branch-1 releases to be in branches-2.y.
>>>
>>> Tracking consideration for when features are available across major
>>> versions fits in well with the "upgrade paths" section in the ref guide.
>>>
>>> We've just gotten in the habit of it only getting filled in when a big
>>> release is coming up.
>>>
>>>
>>>
On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) >>>
Then we must have a upgrade path, for example, 1.5.x can only be
>> upgraded
to 2.2.x if you want all the features still there?
Maybe we should have a release timeline for the first release of all the
minor releases? So when user want to upgrade, they can choose the minor
release which is released later than the current one.
Andrew Purtell 于2019年1月19日 周六13:15写道：
> Also I think branch-1 releases will be done on a monthly cadence
> independent of any branch-2 releases. This is because there are
>> different
> RMs at work with different needs and schedules.
>
> I can certainly help out some with branch-2 releasing if you need it,
> FWIW.
>
> It may also help if we begin talking about 1.x and 2.x as separate
> "products". This can help avoid confusion about features in 1.5 not in
2.1
> but in 2.2. For all practical purposes they are separate products. Some
of
> our community develop and run branch-1. Others develop and run
>> branch-2.
> There is some overlap but the
Re: About how features are integrated to different HBase versions
{quote}
For example, you release 2.1.0 yesterday, 1.5.0 today, and then 2.1.1
tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0 has,
but 2.1.1 should have all features which 1.5.0 has.
{quote}
I don't think can work, normal user mostly won't care about the release
time, they only know 2.1.1 > 2.1.0 > 1.5.0. They will think higher version
includes all the feature in lower version.
I don't get the point why we are now still backporting new features from
branch-2 to branch-1. Yes, there is many 1.x cluster in production, so we
need to release 1.x versions to fix bugs and keep them stable, as the
stable point is still in 1.x.
And at the same time, we should try to move on to 2.x, making branch-1 as a
bugfix branch for sometime before deprecating it. As far as I see, branch-1
is still very 'active', too active I think.
If we stop backport features from branch-2 to branch-1, then there is no
problem, IMHO.
Best Regards
Allan Yang
Andrew Purtell 于2019年1月20日周日 上午5:27写道：
> As branch RM for branch-1 I will always check to make sure a commit there
> has first been committed to branch-2. There will always be an upgrade path
> from a branch-1 based release to a branch-2 based release. The relevant
> JIRAs will either have a 1.x and 2.x fixVersion or the backport JIRA will
> be linked to the one for the branch-2 commit. When making the release notes
> we will be looking at these things (or should, anyway). We can update the
> upgrade paths documentation whenever we find this kind of linkage. Perhaps
> we can describe this for future RMs in the how to release section of the
> doc? Does this satisfy the concerns?
>
> > On Jan 18, 2019, at 11:47 PM, Sean Busbey wrote:
> >
> > I agree with Andrew that we can't both have maintenance releases and
> expect
> > every feature in ongoing branch-1 releases to be in branches-2.y.
> >
> > Tracking consideration for when features are available across major
> > versions fits in well with the "upgrade paths" section in the ref guide.
> >
> > We've just gotten in the habit of it only getting filled in when a big
> > release is coming up.
> >
> >
> >
> >> On Fri, Jan 18, 2019, 23:46 张铎(Duo Zhang) >>
> >> Then we must have a upgrade path, for example, 1.5.x can only be
> upgraded
> >> to 2.2.x if you want all the features still there?
> >>
> >> Maybe we should have a release timeline for the first release of all the
> >> minor releases? So when user want to upgrade, they can choose the minor
> >> release which is released later than the current one.
> >>
> >> Andrew Purtell 于2019年1月19日 周六13:15写道：
> >>
> >>> Also I think branch-1 releases will be done on a monthly cadence
> >>> independent of any branch-2 releases. This is because there are
> different
> >>> RMs at work with different needs and schedules.
> >>>
> >>> I can certainly help out some with branch-2 releasing if you need it,
> >>> FWIW.
> >>>
> >>> It may also help if we begin talking about 1.x and 2.x as separate
> >>> "products". This can help avoid confusion about features in 1.5 not in
> >> 2.1
> >>> but in 2.2. For all practical purposes they are separate products. Some
> >> of
> >>> our community develop and run branch-1. Others develop and run
> branch-2.
> >>> There is some overlap but the overlap is not total. The concerns will
> >>> diverge a bit. I think this is healthy. Everyone is attending to what
> >> they
> >>> need. Let's figure out how to make it work.
> >>>
> On Jan 18, 2019, at 9:04 PM, Andrew Purtell >
> >>> wrote:
>
> Also please be prepared to support forward evolution and maintenance
> of
> >>> branch-1 for, potentially, years. Because it is used in production and
> >> will
> >>> continue to do so for a long time. Features may end up in 1.6.0 that
> only
> >>> appear in 2.3 or 2.4. And in 1.7 that only appear in 2.5 or 2.6. This
> >>> shouldn't be confusing. We just need to document it. JIRA helps some,
> >>> release notes can help a lot more. Maybe in the future a feature to
> >> version
> >>> matrix in the book.
>
> > On Jan 18, 2019, at 8:59 PM, Andrew Purtell <
> andrew.purt...@gmail.com
> >>>
> >>> wrote:
> >
> > This can't work, because we can put things into a new minor that
> >> cannot
> >>> go into a patch relesse. If you say instead 2.2.0 must have everything
> in
> >>> 1.5.0, it can work. The alignment of features should happen at the
> minor
> >>> releases. If we can also have alignment in patch releases too, that
> would
> >>> be great, but can't be mandatory.
> >
> >> On Jan 18, 2019, at 7:12 PM, 张铎(Duo Zhang)
> >>> wrote:
> >>
> >> Please see the red words carefully, I explicitly mentioned that, the
> >>> newer
> >> version should be released LATER, if you want to get all the
> >> features.
> >>
> >> For example, you release 2.1.0 yesterday, 1.5.0 today, and then
> 2.1.1
> >> tomorrow, it is OK that 2.1.0 does not have all feature which 1.5.0
> >>> has,
> >> but 2.1.1 should have all features
Release 2.2.0
Hi, all, there has been six months since we released 2.1.0. And there are 429 issues which fixed version is 2.2.0[1]. Our internal branch which based branch-2 run ITBLL successfully recently. branch-2 is stable now and it is time to release 2.2.0. I volunteered to be the release manager for the 2.2 release line. And plan to cut branch-2.2 from branch-2. For 2.2.0, the biggest change is about AMV2[2]: HBASE-20881 is an incompatible change and different implemenation with branch-2.0 and branch-2.1. Need to add more document about how to rolling upgrade from 2.0.* or 2.1.* to 2.2.*. Meanwhile, need document about how rolling upgrade from 1.* to 2.2.*. Now HBCK2 tool support branch-2's region assignments, too. Another features will be included: 1. HBASE-20610 Procedure V2 - Distributed Log Splitting[3] 2. HBASE-21649 Complete Thrift2[4] 3. HBASE-16707 Improve throttling feature for production usage[5] 4. HBASE-20886 [Auth] Support keytab login in hbase client[6] 5. HBASE-20636 Introduce two bloom filter type : ROWPREFIX_FIXED_LENGTH and ROWPREFIX_DELIMITED[7] Open a issue HBASE-21747 to release 2.2.0[8]. Suggestions are welcomed. Thanks. Best Regards, Guanghao [1] https://issues.apache.org/jira/browse/HBASE-21746?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.2.0 [2] https://issues.apache.org/jira/browse/HBASE-20881 [3] https://issues.apache.org/jira/browse/HBASE-20610 [4] https://issues.apache.org/jira/browse/HBASE-21649 [5] https://issues.apache.org/jira/browse/HBASE-16707 [6] https://issues.apache.org/jira/browse/HBASE-20886 [7] https://issues.apache.org/jira/browse/HBASE-20636 [8] https://issues.apache.org/jira/browse/HBASE-21747
