Re: [DISCUSS] Add Ozone as dependency to hbase-asyncfs ?

[Sean Busbey]

the check that Stephen is referring to is for logic around lease recovery
and not stream flush/sync. the lease recovery is specific to DFS IIRC and
doesn't have a FileSystem marker.

On Wed, Mar 15, 2023 at 3:22 PM Andrew Purtell  wrote:

> So we can test StreamCapabilities in code, in worst case by wrapping some
> probe code during startup with try-catch and examining the exception.
>
> On Wed, Mar 15, 2023 at 1:09 PM Viraj Jasani  wrote:
>
> > As of today, both WAL impl (fshlog and asyncfs) throw
> > StreamLacksCapabilityException if the FS Data OutputStream probe fails
> for
> > Hflush/Hsync:
> >
> > StreamLacksCapabilityException(StreamCapabilities.HFLUSH)
> > and
> > StreamLacksCapabilityException(StreamCapabilities.HSYNC)
> >
> >
> > On Wed, Mar 15, 2023 at 12:51 PM Andrew Purtell 
> > wrote:
> >
> > > Does Hadoop have a marker interface that lets an application know its
> > > FileSystem instances can support hsync/hflush? Ideally all we should
> need
> > > to do is test with instanceof for that marker and use reflection (in
> the
> > > worst case) to get a handle to the hsync or hflush method, and then
> call
> > > it. This approach should be taken wherever we have a requirement to
> use a
> > > special WAL specific API provided by the underlying FileSystem, so we
> can
> > > abstract it sufficiently to not require a direct dependency on Ozone or
> > S3A
> > > or any non HDFS filesystem.
> > >
> > > On Wed, Mar 15, 2023 at 12:31 PM Tak Lon (Stephen) Wu <
> tak...@apache.org
> > >
> > > wrote:
> > >
> > > > Hi team,
> > > >
> > > > Recently, Wei-Chiu and I have been discussing about if HBase can use
> > > > Ozone as another storage as WAL (see the hsync and hflush JIRAs [1])
> > > > and HFile, for HFile it’s pluggable by configuring the file system to
> > > > use Ozone File System (Ozone)
> > > >
> > > > But we found that the WAL it’s a bit different, especially
> > > > RecoverLeaseFSUtils#recoverFileLease [2], it has one check about if
> > > > the file system is an instance of HDFS, and thus WAL recovery to
> > > > execute file lease recovery from RS crashes. Here, if we would like
> to
> > > > add Ozone, it does not matter by importing as a direct dependency to
> > > > perform similar lease recovery or via reflection by class name in
> > > > plaintext String, we still need to somehow introduce Ozone to be
> > > > another supported file system. (we can discuss how we can implement
> > > > better as well)
> > > >
> > > > We also found other places e.g. FSUtils and HFileSystem have used
> > > > DistributedFileSystem, but it should be able to move them into either
> > > > hbase-asyncfs or a new FS related component to separate the use of
> > > > different supported file systems.
> > > >
> > > > So, we’re wondering if anyone would have any objections to adding
> > > > Ozone as a dependency to hbase-asyncfs? or if you have a better idea
> > > > how this could be added without adding Ozone as dependency, please
> > > > feel free to comment on this thread.
> > > >
> > > >
> > > > [1] Ozone is working on support for hsync and hflush,
> > > > https://issues.apache.org/jira/browse/HDDS-7593,
> > > > https://issues.apache.org/jira/browse/HDDS-4353
> > > > [2] RecoverLeaseFSUtils#recoverFileLease,
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/blob/master/hbase-asyncfs/src/main/java/org/apache/hadoop/hbase/util/RecoverLeaseFSUtils.java#L53-L63
> > > >
> > > > Thanks,
> > > > Stephen
> >
>

[jira] [Created] (HBASE-27425) Run flaky test job more often

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27425:
---

 Summary: Run flaky test job more often
 Key: HBASE-27425
 URL: https://issues.apache.org/jira/browse/HBASE-27425
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Sean Busbey
 Attachments: ci-hbase-hbase-flaky-test-master-time-png.png, 
ci-hbase-hbase-load-chart-long.png

We currently run the flaky tests once per 4 hours and keep a backlog of 50 
runs. That means it takes a perfect test a bit over 8 days to get back out of 
being penalized.

we regularly are leaving a bunch of executors idle (see attached PNG) and I 
think we should run the tests more often.

we use an executor per branch for a flaky test run. the duration depends on the 
specific tests that are flagged, but it has a timeout of 2 hours.

as an example the master branch tests currently take < 30 minutes (see other 
attached PNG).





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] HBase 2.5 / Hadoop 3 artifacts

[Sean Busbey]

I think changing the default hadoop profile for builds in branch-2 would
unnecessarily complicate our compatibility messaging so long as Hadoop 2
hasn't gone EOL.

On Mon, Aug 29, 2022 at 5:30 AM Nick Dimiduk  wrote:

> Should we also make hadoop3 the default active profile for branch-2 going
> forward?
>
> On Fri, Aug 26, 2022 at 5:25 PM Andrew Purtell 
> wrote:
>
> > The security posture of Hadoop 2 in general is a problem, because
> > maintenance on that branch is spotty, that is just how it goes. We had
> the
> > same situation with our now EOL branch-1. I know Hadoop released 2.10.2
> to
> > address some CVE worthy problems but it is unclear if 2.10.2 addresses
> all
> > known issues, unlike 3.3.4. Also as you know Hadoop 2 has unpatchable
> > dependencies on org.codehaus versions of Jackson and Jetty, which
> > themselves have high scoring CVEs that will never be fixed because they
> are
> > EOL, and other similar issues. Hadoop 3 doesn’t completely solve such
> > problems but is the only realistic place we can hope they can be
> addressed
> > as required. For organizations that implement or require a top to bottom
> > security audit of their software bill of materials, it seems possible to
> > avoid user pain by providing supported convenience artifacts *and*
> > libraries built against Hadoop 3 APIs in the Apache repository
> addressable
> > with a Maven classifier.
> >
> > My employer has some interests in this area that align so I would like to
> > sponsor (implement, review, commit, RM backfill releases, etc.) this
> work.
> > Would there be any objections? Read through the thread for some thoughts
> on
> > approach. Summarized:
> >
> > - Amend create-release to build, stage, and deploy a -hadoop3 variant
> > build by activating the Hadoop 3 build profile.
> >
> > - Amend the Hadoop 3 build profile to flatten POMs before deployment to
> > resolve potential downstream issues due to Hadoop 3 being a non-default
> > build profile. (This could also be applied to all builds.)
> >
> > - Amend hbase-vote to be aware of and evaluate if present -hadoop3
> variant
> > artifacts.
> >
> >
> > > On Aug 25, 2022, at 10:40 AM, Andrew Purtell  >
> > wrote:
> > >
> > > Thanks, that would work.
> > >
> > >> On Aug 25, 2022, at 11:35 AM, Sean Busbey  wrote:
> > >>
> > >> yes, the flatten plugin. We use it in hbase-connectors already.
> > >>
> > >> https://www.mojohaus.org/flatten-maven-plugin/
> > >>
> > >> this sounds like it could also be a use case for BOMs, which would
> also
> > >> benefit users of our client artifacts that use build tools that don't
> > >> respect maven profiles generally, like gradle.
> > >>
> > >>> On Thu, Aug 25, 2022 at 10:30 AM Andrew Purtell <
> > andrew.purt...@gmail.com>
> > >>> wrote:
> > >>>
> > >>> Thinking about this a bit more, we will have an issue in that the
> POMs
> > >>> published from our -hadoop3 build will not have a default activation
> > of our
> > >>> Hadoop 3 build profile. The convenience binaries will function as
> > expected
> > >>> but Maven will read and process eg Phoenix POMs, then download and
> > perform
> > >>> substitutions on HBase POMs, and then etc, so downstreamers like
> > Phoenix
> > >>> will have to set up the hadoop.profile variable for us in their
> default
> > >>> build profile or else the transitive paths through us may be wrong. I
> > >>> wonder if there is a Maven plugin available for deploying POMs with
> all
> > >>> variable substitutions performed before deployment, that would solve
> > that
> > >>> problem and all conceivable related issues.
> > >>>
> > >>>> On Aug 25, 2022, at 11:03 AM, Andrew Purtell <
> > andrew.purt...@gmail.com>
> > >>> wrote:
> > >>>>
> > >>>> I think 2.x is going to have a few years of life remaining so it
> > would
> > >>> be best, if we are going to address this, to have a 2.x solution was
> > well
> > >>> as a 3.x one.
> > >>>>
> > >>>> In my opinion we can continue to publish 2.4 and 2.5 (and 2.6)
> > unchanged
> > >>> and then also introduce a Hadoop 3 release using “hadoop3” or similar
> > as
> > >>> Maven classifier. Phoenix could specify this classifier in 

[jira] [Resolved] (HBASE-24457) release scripts on mac os are too slow

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-24457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-24457.
-
Fix Version/s: 3.0.0-alpha-1
   (was: 3.0.0-alpha-4)
   Resolution: Fixed

> release scripts on mac os are too slow
> --
>
> Key: HBASE-24457
> URL: https://issues.apache.org/jira/browse/HBASE-24457
> Project: HBase
>  Issue Type: Umbrella
>  Components: community
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Critical
> Fix For: 3.0.0-alpha-1
>
>
> related changes  to try to bring the time down to something I can use



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] HBase 2.5 / Hadoop 3 artifacts

[Sean Busbey]

yes, the flatten plugin. We use it in hbase-connectors already.

https://www.mojohaus.org/flatten-maven-plugin/

this sounds like it could also be a use case for BOMs, which would also
benefit users of our client artifacts that use build tools that don't
respect maven profiles generally, like gradle.

On Thu, Aug 25, 2022 at 10:30 AM Andrew Purtell 
wrote:

> Thinking about this a bit more, we will have an issue in that the POMs
> published from our -hadoop3 build will not have a default activation of our
> Hadoop 3 build profile. The convenience binaries will function as expected
> but Maven will read and process eg Phoenix POMs, then download and perform
> substitutions on HBase POMs, and then etc, so downstreamers like Phoenix
> will have to set up the hadoop.profile variable for us in their default
> build profile or else the transitive paths through us may be wrong. I
> wonder if there is a Maven plugin available for deploying POMs with all
> variable substitutions performed before deployment, that would solve that
> problem and all conceivable related issues.
>
> > On Aug 25, 2022, at 11:03 AM, Andrew Purtell 
> wrote:
> >
> > I think 2.x is going to have a few years of life remaining so it would
> be best, if we are going to address this, to have a 2.x solution was well
> as a 3.x one.
> >
> > In my opinion we can continue to publish 2.4 and 2.5 (and 2.6) unchanged
> and then also introduce a Hadoop 3 release using “hadoop3” or similar as
> Maven classifier. Phoenix could specify this classifier in their POMs.
> Everyone should be happy. Users who already are comfortable with the Hadoop
> 2 default don’t have to change anything. A one time POM change on the
> Phoenix side is required but that’s it.
> >
> > The additional build time complexity for generating two releases can be
> incorporated into create-release. Nobody does manual releases any more as
> far as I know. Likewise, download and verification of -hadoop3 convenience
> binaries can be added to hbase-vote. I believe we are all using that tool
> for verification of releases now. After these one time changes are landed
> the cost for RMs and PMC will be only in a roughly doubled amount of time
> needed to build and verify releases.
> >
> >> On Aug 17, 2022, at 9:06 AM, Nick Dimiduk  wrote:
> >>
> >> Hi Geoffrey,
> >>
> >> I have no complaints with shipping convenience binaries built against
> both
> >> Hadoop2 and Hadoop3. The primary challenge is implementing the
> >> necessary build changes, the secondary challenge is verifying/testing it
> >> works reliably.
> >>
> >> But for Phoenix, are you asking for convenience binaries, or are you
> asking
> >> for artifacts published into maven that have the Hadoop3 profile
> activated
> >> and specify the associated dependencies?
> >>
> >> I'm afraid that the 2.5.0 release ship has already sailed. I've heard
> talk
> >> of a 2.6 "fast-follow", so maybe someone can have the build changes
> ready
> >> for that? Also, isn't this a too little, too late situation? Shouldn't
> we
> >> shift our focus to releasing 3.0, which has dropped support for Hadoop2?
> >>
> >> Thanks,
> >> Nick
> >>
>  On Tue, Aug 16, 2022 at 9:30 PM Geoffrey Jacoby 
> wrote:
> >>>
> >>> I see that the next HBase 2.5 RC is imminent, and before that's set in
> >>> stone, I wanted to bring up the question of whether there will be
> official
> >>> HBase 2.5 binaries built with the Hadoop 3 profile and available in the
> >>> usual Maven repositories. (In addition to the usual Hadoop 2 profile
> >>> binaries)
> >>>
> >>> The HBase 2.x line has a commitment to maintain support for Hadoop
> 2.x, but
> >>> Hadoop 3.3 is the current stable Hadoop line and the most recent
> release
> >>> notes [1] encourage all users of Hadoop  2.x to upgrade to Hadoop 3.
> >>>
> >>> Without convenience artifacts built against Hadoop 3, no end-users with
> >>> Hadoop 3 clusters will be able to use the Apache-distributed binaries
> and
> >>> will instead have to recompile HBase from source themselves, or use a
> 3rd
> >>> party distribution that does so for them.
> >>>
> >>> This is especially inconvenient for downstream projects such as Apache
> >>> Phoenix, which has never  officially supported the HBase 2.x / Hadoop
> 2.10
> >>> combination. (It currently supports only HBase 2.3 or 2.4 with Hadoop
> 3.
> >>> HBase 2.5 support will be added very shortly after its release as part
> of
> >>> Phoenix 5.2.)
> >>>
> >>> To even run the Phoenix IT tests locally requires contributors to
> download
> >>> the HBase source release and manually mvn install to their local maven
> repo
> >>> using the Hadoop 3 profile, to avoid crashes in the HBase
> minicluster.[2]
> >>> This is a barrier to new contributors and confuses even veteran ones,
> and
> >>> has to be done again for every new HBase release.
> >>>
> >>> In general, I expect the Hadoop 3 user base to grow and the Hadoop 2.10
> >>> user base to shrink with every future HBase 2 release, so I think this
> is 

[jira] [Resolved] (HBASE-26983) Upgrade JRuby to 9.3.4.0

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26983.
-
Resolution: Fixed

> Upgrade JRuby to 9.3.4.0
> 
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15 
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed 
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
>   initialize at org/jruby/RubyIO.java:1015
>         open at org/jruby/RubyIO.java:1156
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
>   initialize at uri:classloader:/irb/hirb.rb:49
>        at classpath:/jar-bootstrap.rb:223
> {code}
>  
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27106) HBase site generation should use its own JRuby

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27106:
---

 Summary: HBase site generation should use its own JRuby
 Key: HBASE-27106
 URL: https://issues.apache.org/jira/browse/HBASE-27106
 Project: HBase
  Issue Type: Bug
  Components: website
Reporter: Sean Busbey
Assignee: Sean Busbey


The maven plugin we use to generate the reference guide, 
{{asciidoctor-maven-plugin}}, does not yet work with JRuby 9.3.

Due to a dependency error we currently get a mix of JRuby 9.2 and 9.3 libraries 
when building the site goal, which fails without an actionable error message. 
Correcting the dependency to consistently get JRuby 9.3 also results in 
failure, but due to gem resolution.

We originally started overriding the JRuby version to get Aarch64 support. The 
version that ships with the plugin has this support already.

Falling back to JRuby 9.2 for the plugin will mean our reference guide building 
won't work on M1 systems, but I don't see how we can change that until the 
plugin updates for JRuby 9.3 changes.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Reopened] (HBASE-26983) Upgrade JRuby to 9.3.4.0

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-26983:
-

reopened to evaluate impact on site build

> Upgrade JRuby to 9.3.4.0
> 
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15 
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed 
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
>   initialize at org/jruby/RubyIO.java:1015
>         open at org/jruby/RubyIO.java:1156
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
>   initialize at uri:classloader:/irb/hirb.rb:49
>        at classpath:/jar-bootstrap.rb:223
> {code}
>  
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (HBASE-26983) Upgrade JRuby to 9.3.4.0

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26983.
-
Release Note: 
Updates the version of JRuby that ships with HBase for the HBase shell to 
9.3.4.0.

Note that this changes the supported version of Ruby for HBase shell 
integration to Ruby 2.6.
  Resolution: Fixed

> Upgrade JRuby to 9.3.4.0
> 
>
> Key: HBASE-26983
> URL: https://issues.apache.org/jira/browse/HBASE-26983
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Affects Versions: 3.0.0-alpha-2, 2.4.11
> Environment: Apple M1 OSX ARM64.
>Reporter: Vijay Akkineni
>Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> Hbase shell is failing to start on Apple M1 OSX ARM 64 processor architecture.
> *Error:*
> {code}
> Version 2.4.11, r7e672a0da0586e6b7449310815182695bc6ae193, Tue Mar 15 
> 10:31:00 PDT 2022
> Took 0.0010 seconds
> NotImplementedError: fstat unimplemented unsupported or native support failed 
> to load; see https://github.com/jruby/jruby/wiki/Native-Libraries
>   initialize at org/jruby/RubyIO.java:1015
>         open at org/jruby/RubyIO.java:1156
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/input-method.rb:141
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/context.rb:70
>   initialize at 
> uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb.rb:410
>   initialize at uri:classloader:/irb/hirb.rb:49
>        at classpath:/jar-bootstrap.rb:223
> {code}
>  
> {*}Uname output{*}:
> {code}
> Darwin vijays-mbp.lan 21.4.0 Darwin Kernel Version 21.4.0: Fri Mar 18 
> 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000 arm64
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Resolved] (HBASE-27006) cordon off large ci worker nodes

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-27006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-27006.
-
Fix Version/s: 2.5.0
   2.6.0
   3.0.0-alpha-3
   2.4.13
 Assignee: Sean Busbey
   Resolution: Fixed

> cordon off large ci worker nodes
> 
>
> Key: HBASE-27006
> URL: https://issues.apache.org/jira/browse/HBASE-27006
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.13
>
>
> Make sure our node labeling keeps test workloads that off of the new large 
> spec nodes so that they're only used for the nightly k8s based tests



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27012) nightly ITBLL

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27012:
---

 Summary: nightly ITBLL
 Key: HBASE-27012
 URL: https://issues.apache.org/jira/browse/HBASE-27012
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27010) container images for HBase

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27010:
---

 Summary: container images for HBase
 Key: HBASE-27010
 URL: https://issues.apache.org/jira/browse/HBASE-27010
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


we need container images of the build that got made during the nightly run



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27011) migrate existing nightly test

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27011:
---

 Summary: migrate existing nightly test
 Key: HBASE-27011
 URL: https://issues.apache.org/jira/browse/HBASE-27011
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


once we can run a cluster on k8s in the nightly worker node, migrate our 
existing test to use it.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27009) container images for ZooKeeper

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27009:
---

 Summary: container images for ZooKeeper
 Key: HBASE-27009
 URL: https://issues.apache.org/jira/browse/HBASE-27009
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


we need image(s) for a zk cluster for our nightly cluster run



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27008) container images for Hadoop

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27008:
---

 Summary: container images for Hadoop
 Key: HBASE-27008
 URL: https://issues.apache.org/jira/browse/HBASE-27008
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


we need image(s) for the hadoop cluster we run the nightly tests on top of



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27007) nightly support scripts for install/setup of single-node k8s env

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27007:
---

 Summary: nightly support scripts for install/setup of single-node 
k8s env
 Key: HBASE-27007
 URL: https://issues.apache.org/jira/browse/HBASE-27007
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


minikube , kind, something like that.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27006) cordon off large ci worker nodes

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27006:
---

 Summary: cordon off large ci worker nodes
 Key: HBASE-27006
 URL: https://issues.apache.org/jira/browse/HBASE-27006
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


Make sure our node labeling keeps test workloads that off of the new large spec 
nodes so that they're only used for the nightly k8s based tests



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-27005) move nightly cluster tests to run on k8s

[Sean Busbey (Jira)]

Sean Busbey created HBASE-27005:
---

 Summary: move nightly cluster tests to run on k8s
 Key: HBASE-27005
 URL: https://issues.apache.org/jira/browse/HBASE-27005
 Project: HBase
  Issue Type: Umbrella
  Components: integration tests
Reporter: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (HBASE-26936) project Jenkins node needs a fail-safe clean up of surefire deferred output

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26936:
---

 Summary: project Jenkins node needs a fail-safe clean up of 
surefire deferred output
 Key: HBASE-26936
 URL: https://issues.apache.org/jira/browse/HBASE-26936
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Sean Busbey
Assignee: Sean Busbey


We've had build nodes run out of space a few times now caused by surefire 
deferred stdout / stderr filling a disk.

 

Come up with a strategy to make recovery from this automated.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26889) nightly yetus tests don't reflect failure when required environment variables are missing.

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26889:
---

 Summary: nightly yetus tests don't reflect failure when required 
environment variables are missing.
 Key: HBASE-26889
 URL: https://issues.apache.org/jira/browse/HBASE-26889
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Sean Busbey


the nightly yetus wrapper shows an error when we're missing some needed 
environment variables, but the stage shows success still.

e.g
{code:java}
[2022-03-24T22:35:39.450Z] [ERROR] Required environment variable 'DEBUG' is not 
set.
[2022-03-24T22:35:39.451Z] [ERROR] Required environment variable 
'USE_YETUS_PRERELEASE' is not set.
[2022-03-24T22:35:39.451Z] [ERROR] Please set the required environment 
variables before invoking. If this error is  on Jenkins, then please file a 
JIRA about the error. {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26888) Update branch-specific nightly test handling to allow for earlier release line feature branches

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26888:
---

 Summary: Update branch-specific nightly test handling to allow for 
earlier release line feature branches
 Key: HBASE-26888
 URL: https://issues.apache.org/jira/browse/HBASE-26888
 Project: HBase
  Issue Type: Task
  Components: integration tests, test
Reporter: Sean Busbey


due to limitations in ASF infra our branch protection rules prohibit force 
pushes or deletions on branches that start with "master" or "branch-".

Our current nightly tests assume that feature branches specific to branch-2 or 
branch-1 will start with the release branch name rather than the feature Jira 
key. However, doing so would mean we need to file INFRA jiras to clean up 
afterwards.

 

We can't change the limitations of branch protection rules, so we should update 
the nightly tests to allow for feature branches named like 
{{{}HBASE-X-branch-2{}}}.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26887) nightly integration test claims success while showing error parsing shell output

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26887:
---

 Summary: nightly integration test claims success while showing 
error parsing shell output
 Key: HBASE-26887
 URL: https://issues.apache.org/jira/browse/HBASE-26887
 Project: HBase
  Issue Type: Bug
  Components: integration tests, shell
Reporter: Sean Busbey


current master branch claims that the integration test is succeeding, but 
looking at the detailed output shows a failure to parse the shell output:

 
{code:java}
Starting up Hadoop
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
Verifying configs
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/output-integration/hadoop-3/hbase-conf/core-site.xml:
 valid
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/output-integration/hadoop-3/hbase-conf/hbase-site.xml:
 valid
OK
Listing HDFS contents
Starting up HBase
running master, logging to 
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/hbase-install/bin/../logs/hbase-jenkins-master-jenkins-hbase12.out
retry waiting for hbase to come up.
Setting up table 'test:example' with 1,000 regions
writing out example TSV to example.tsv
uploading example.tsv to HDFS
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Importing TSV via shaded client artifact for HBase - MapReduce integration.
Verifying row count from import.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
 line 418: [: hbase:002:0> : integer expression expected
Hadoop client jars not given; getting them from 'hadoop classpath' for the 
example.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Building shaded client example.
Running shaded client example. It'll fetch the set of regions, round-trip them 
to a file in HDFS, then write them one-per-row into the test table.
Checking on results of example program.
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Verifying row count from example.
/home/jenkins/jenkins-home/workspace/HBase_Nightly_master/component/dev-support/hbase_nightly_pseudo-distributed-test.sh:
 line 531: [: hbase:002:0> : integer expression expected
ERROR: Only found hbase:002:0>  rows.
Shutting down HBase {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] EOM 1.7 and branch-1? (proposal: 2022/07/21)

[Sean Busbey]

Sorry for the lack of clarity. I meant to include that I’ll send the
message to the user list later today if folks don’t have a specific concern
with me doing so.

On Sat, Mar 19, 2022 at 9:03 AM Sean Busbey  wrote:

> I think it’d be worth checking with user@ to see if there are continued
> users of branch-1 that would be interested in getting more involved to keep
> the branch going with eg dependency updates and security fixes should they
> be needed.
>
>
>
> On Fri, Mar 18, 2022 at 11:43 PM Viraj Jasani  wrote:
>
>> +1 for the EOM of branch-1. We occasionally commit few minor fixes if
>> required, beyond that it doesn’t get much attention.
>> Perhaps no need to wait for one year anniversary of 1.7 release if we have
>> sufficient votes to retire branch-1 sooner?
>>
>>
>> On Fri, 18 Mar 2022 at 3:06 AM, Andrew Purtell 
>> wrote:
>>
>> > We previously discussed this topic on this thread:
>> >
>> >   https://lists.apache.org/thread/hlp18jjjxxpf62spd8zkhmht23hmpljg
>> >
>> > Is it time to consider EOL of branch-1 and all 1.x releases ?
>> >
>> > There doesn't seem to be much developer interest in branch-1 beyond
>> > occasional maintenance. This is understandable. Per our compatibility
>> > guidelines, branch-1 commits must be compatible with Java 7, and the
>> range
>> > of acceptable versions of third party dependencies is also restricted
>> due
>> > to Java 7 compatibility requirements. Most developers are writing code
>> with
>> > Java 8+ idioms these days. For that reason and because the branch-1 code
>> > base is generally aged at this point, all but trivial (or lucky!)
>> backports
>> > require substantial changes in order to integrate adequately. Let me
>> also
>> > observe that branch-1 artifacts are not fully compatible with Java 11 or
>> > later.
>> >
>> > It is my more recent observation that relatively little maintenance
>> > activity is occurring with respect to branch-1.
>> >
>> > The last release from branch-1 was 1.7.1, on 2021/07/21.
>> >
>> > If there are no more 1.x releases in the meantime, shall we use the
>> > occasion of the one year anniversary of this last 1.x release and
>> announce
>> > EOM of HBase 1.7 and all of branch-1 on or about 2022/07/21 ?
>> >
>> > --
>> > Best regards,
>> > Andrew
>> >
>>
>

Re: [DISCUSS] EOM 1.7 and branch-1? (proposal: 2022/07/21)

[Sean Busbey]

I think it’d be worth checking with user@ to see if there are continued
users of branch-1 that would be interested in getting more involved to keep
the branch going with eg dependency updates and security fixes should they
be needed.



On Fri, Mar 18, 2022 at 11:43 PM Viraj Jasani  wrote:

> +1 for the EOM of branch-1. We occasionally commit few minor fixes if
> required, beyond that it doesn’t get much attention.
> Perhaps no need to wait for one year anniversary of 1.7 release if we have
> sufficient votes to retire branch-1 sooner?
>
>
> On Fri, 18 Mar 2022 at 3:06 AM, Andrew Purtell 
> wrote:
>
> > We previously discussed this topic on this thread:
> >
> >   https://lists.apache.org/thread/hlp18jjjxxpf62spd8zkhmht23hmpljg
> >
> > Is it time to consider EOL of branch-1 and all 1.x releases ?
> >
> > There doesn't seem to be much developer interest in branch-1 beyond
> > occasional maintenance. This is understandable. Per our compatibility
> > guidelines, branch-1 commits must be compatible with Java 7, and the
> range
> > of acceptable versions of third party dependencies is also restricted due
> > to Java 7 compatibility requirements. Most developers are writing code
> with
> > Java 8+ idioms these days. For that reason and because the branch-1 code
> > base is generally aged at this point, all but trivial (or lucky!)
> backports
> > require substantial changes in order to integrate adequately. Let me also
> > observe that branch-1 artifacts are not fully compatible with Java 11 or
> > later.
> >
> > It is my more recent observation that relatively little maintenance
> > activity is occurring with respect to branch-1.
> >
> > The last release from branch-1 was 1.7.1, on 2021/07/21.
> >
> > If there are no more 1.x releases in the meantime, shall we use the
> > occasion of the one year anniversary of this last 1.x release and
> announce
> > EOM of HBase 1.7 and all of branch-1 on or about 2022/07/21 ?
> >
> > --
> > Best regards,
> > Andrew
> >
>

Posting EOM to user-zh

[Sean Busbey]

Could someone translate the 2.3 EOM notice for the user-zh mailing list?

Longer term we should track a template that can get filled in for each of
the user lists so the poster needn’t be particularly proficient in either
language.

[jira] [Resolved] (HBASE-26704) EOL HBase 2.3.x

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26704.
-
Resolution: Fixed

> EOL HBase 2.3.x
> ---
>
> Key: HBASE-26704
> URL: https://issues.apache.org/jira/browse/HBASE-26704
> Project: HBase
>  Issue Type: Task
>  Components: community
>    Reporter: Sean Busbey
>Priority: Major
>
> VOTE thread is here:
> [https://lists.apache.org/thread/v3gfoyf255rpvql4h3g68ong5p4ps4rs]
>  * remove from [https://hbase.apache.org/downloads]
>  * delete from dist svn repo 
> [https://dist.apache.org/repos/dist/release/hbase/] (which will remove the 
> release from downloads.apache.org)
>  * send an announcement to user@hbase and dev@hbase
>  * delete Jenkins jobs



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[ANNOUNCE] end of maintenance (EOM) for Apache HBase 2.3 releases

[Sean Busbey]

Hi folks!

The Apache HBase 2.3 release line reached the end of maintenance in
October 2021. Our apologies for the late notification.

The final release was 2.3.7 and is available from archive.apache.org.
No further releases from branch-2.3 are planned.

Users of HBase 2.3.z should upgrade to the current stable release
(currently 2.4.10) as quickly as possible.

Thanks to all the contributors who helped make HBase 2.3 possible over
its lifespan.

Re: [DISCUSS] ci-hbase capacity and what we'd like to see tested

[Sean Busbey]

It sounds like we have consensus on the general approach.

I'll file some jiras for the work. I have some time to put towards
implementing things. If folks want to help move things along faster
they're welcome to pitch in.

On Sun, Mar 13, 2022 at 1:35 PM Andrew Purtell  wrote:
>
> I like the idea of multi node testing, because the mini cluster does an 
> admirable job but cannot truly emulate a production deploy because of various 
> singletons in our code or that of our dependencies. It would also be pretty 
> nice if k8s was the substrate — and ultimately is used to inject chaos too 
> (via hbase-it) — because it would help us detect if we violate required 
> discipline for that common deployment target, like inappropriate caching of 
> DNS resolutions concurrent with pod cycling, to pick a historical example.
>
> Even just periodic execution of ITBLL would be nice.
>
> So I guess the next question is what does that require of us, the larger 
> community. Who proposed the work? Who performs it? We should open some JIRAs 
> to kick things off?
>
> > On Mar 10, 2022, at 8:32 AM, Sean Busbey  wrote:
> >
> > Hi folks!
> >
> > Quick background: all of the automated testing for nightly and PR
> > contributions is now running on a dedicated Jenkins instance (
> > ci-hbase.apache.org ). We moved our existing 10 dedicated nodes off of
> > the ci-hadoop controller and thanks to a new anonymous donor we were
> > able to add an additional 10 nodes.
> >
> > The new donor gave enough of a contribution that we can make some
> > decisions as a community about expanding these resources further.
> >
> > The new 10 nodes run 2 executors each (same as our old nodes), have
> > this shape, and are considered "medium" by the provider we're getting
> > them from:
> >
> > 64GB DDR4 ECC RAM
> > Intel® Xeon® E-2176G hexa-core processor with Hyper-Threading Coffee Lake.
> > 2 x 960 GB NVMe SSD Datacenter Edition (RAID 1)
> >
> > To give an idea of what the current testing workload of our project
> > looks like, we can use the built in jenkins utilization tooling for
> > our general purpose label 'hbase'[0].
> >
> > If you look at the last 4 days of utilization[1] we have a couple of
> > periods with a small backlog of ~2 executors worth of work. The
> > measurements are very rolled up so it's hard to tell specifics. On the
> > chart of the last day or so[2] we can see two periods of 1-2 hours
> > where we have a backlog of 2-4 executors worth of work.
> >
> > for comparison, the chart for immediately after we had to burn off ~3
> > days of backlog because our worker nodes were offline back at the end
> > of february shows no queue[3].
> >
> > I think we could possibly benefit from adding 1-2 additional medium
> > worker nodes, but the long periods where we have ~half our executors
> > idle makes me think some refactoring or timing changes would maybe be
> > a better way to improve our current steady state workload.
> >
> > One thing that we currently lack is robust integration testing of a
> > cluster deployment. At the moment our nightly jobs spin up a test that
> > makes a single node version of Hadoop and then a single node hbase on
> > top of it. It then does a trivial functionality test[4].
> >
> > The host provider we use for jenkins worker nodes has a large node shaped 
> > like:
> > 160GB RAM
> > Intel® Xeon® W-2295 18-Core Cascade-Lake W Hyper-Threading
> > 2 x 960GB NVMe drives as RAID1
> >
> > A pretty short path to improvement would be if we got 1 or 2 of these
> > nodes and moved our integration test to use the minikube project[5] to
> > run a local kubernetes environment. We could then deploy a small but
> > multinode Hadoop and HBase cluster and run e.g. ITBLL against it in
> > addition to whatever checking of cli commands, shell expectations,
> > etc.
> >
> > What do y'all think?
> >
> > [0]: https://ci-hbase.apache.org/label/hbase/load-statistics
> > [1]: 
> > https://issues.apache.org/jira/secure/attachment/13040941/ci-hbase-long-graph-20220310.png
> > [2]: 
> > https://issues.apache.org/jira/secure/attachment/13040940/ci-hbase-medium-graph-20220310.png
> > [3]: 
> > https://issues.apache.org/jira/secure/attachment/13040939/ci-hbase-medium-graph-20220223.png
> > [4]: 
> > https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh
> > [5]: https://minikube.sigs.k8s.io/docs/

[DISCUSS] ci-hbase capacity and what we'd like to see tested

[Sean Busbey]

Hi folks!

Quick background: all of the automated testing for nightly and PR
contributions is now running on a dedicated Jenkins instance (
ci-hbase.apache.org ). We moved our existing 10 dedicated nodes off of
the ci-hadoop controller and thanks to a new anonymous donor we were
able to add an additional 10 nodes.

The new donor gave enough of a contribution that we can make some
decisions as a community about expanding these resources further.

The new 10 nodes run 2 executors each (same as our old nodes), have
this shape, and are considered "medium" by the provider we're getting
them from:

64GB DDR4 ECC RAM
Intel® Xeon® E-2176G hexa-core processor with Hyper-Threading Coffee Lake.
2 x 960 GB NVMe SSD Datacenter Edition (RAID 1)

To give an idea of what the current testing workload of our project
looks like, we can use the built in jenkins utilization tooling for
our general purpose label 'hbase'[0].

If you look at the last 4 days of utilization[1] we have a couple of
periods with a small backlog of ~2 executors worth of work. The
measurements are very rolled up so it's hard to tell specifics. On the
chart of the last day or so[2] we can see two periods of 1-2 hours
where we have a backlog of 2-4 executors worth of work.

for comparison, the chart for immediately after we had to burn off ~3
days of backlog because our worker nodes were offline back at the end
of february shows no queue[3].

I think we could possibly benefit from adding 1-2 additional medium
worker nodes, but the long periods where we have ~half our executors
idle makes me think some refactoring or timing changes would maybe be
a better way to improve our current steady state workload.

One thing that we currently lack is robust integration testing of a
cluster deployment. At the moment our nightly jobs spin up a test that
makes a single node version of Hadoop and then a single node hbase on
top of it. It then does a trivial functionality test[4].

The host provider we use for jenkins worker nodes has a large node shaped like:
160GB RAM
Intel® Xeon® W-2295 18-Core Cascade-Lake W Hyper-Threading
2 x 960GB NVMe drives as RAID1

A pretty short path to improvement would be if we got 1 or 2 of these
nodes and moved our integration test to use the minikube project[5] to
run a local kubernetes environment. We could then deploy a small but
multinode Hadoop and HBase cluster and run e.g. ITBLL against it in
addition to whatever checking of cli commands, shell expectations,
etc.

What do y'all think?

[0]: https://ci-hbase.apache.org/label/hbase/load-statistics
[1]: 
https://issues.apache.org/jira/secure/attachment/13040941/ci-hbase-long-graph-20220310.png
[2]: 
https://issues.apache.org/jira/secure/attachment/13040940/ci-hbase-medium-graph-20220310.png
[3]: 
https://issues.apache.org/jira/secure/attachment/13040939/ci-hbase-medium-graph-20220223.png
[4]: 
https://github.com/apache/hbase/blob/master/dev-support/hbase_nightly_pseudo-distributed-test.sh
[5]: https://minikube.sigs.k8s.io/docs/

[jira] [Created] (HBASE-26823) ci-hbase expansion

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26823:
---

 Summary: ci-hbase expansion
 Key: HBASE-26823
 URL: https://issues.apache.org/jira/browse/HBASE-26823
 Project: HBase
  Issue Type: Task
  Components: community, integration tests, test
Reporter: Sean Busbey
Assignee: Sean Busbey


I need a place to host images related to a dev@hbase discussion about how 
ci-hbase is going an what we as a project would like to expand into.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26546) hbase-shaded-client missing required thirdparty classes under hadoop 3.3.1

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26546.
-
Resolution: Fixed

> hbase-shaded-client missing required thirdparty classes under hadoop 3.3.1
> --
>
> Key: HBASE-26546
> URL: https://issues.apache.org/jira/browse/HBASE-26546
> Project: HBase
>  Issue Type: Bug
>  Components: Client, hadoop3, shading
>Affects Versions: 2.4.3, 2.4.4, 2.4.5, 2.4.6, 2.4.7, 2.4.8, 2.4.9
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.10
>
>
> In HBASE-25792, the shaded thirdparty libraries from hadoop were removed from 
> the hbase-shaded-client fat jar to satisfy invariant checks. Unfortunately 
> this causes users of hbase-shaded-client to fail, because required classes 
> are not available at runtime.
> The specific failure I'm seeing is when trying to call new Configuration(), 
> which results in:
>  
>  
> {code:java}
> Caused by: java.lang.NoClassDefFoundError: 
> org/apache/hadoop/thirdparty/com/google/common/base/Preconditions   
>   at 
> org.apache.hadoop.conf.Configuration$DeprecationDelta.(Configuration.java:430)
>    
>   at 
> org.apache.hadoop.conf.Configuration$DeprecationDelta.(Configuration.java:443)
>    
>   at 
> org.apache.hadoop.conf.Configuration.(Configuration.java:525){code}
>  
>  
> If you take a look at the hbase-shaded-client fat jar, it contains the 
> org.apache.hadoop.conf.Configuration class as you'd expect. If you decompile 
> that class (or look at the 3.3.1 source), you'll see that there is an import 
> for org.apache.hadoop.thirdparty.com.google.common.base.Preconditions but the 
> fat jar does not provide it.
>  
> One way for clients to get around this is to add an explicit dependency on 
> hadoop-shaded-guava, but this is problematic for a few reasons:
>  
> - it's best practice to use maven-dependency-plugin to disallow declared, 
> unused dependencies (which this would be)
> - it requires users to continually keep the version of hadoop-shaded-guava 
> up-to-date over time.
> - it only covers guava, but there is also protobuf and potentially other 
> shaded libraries in the future.
>  
> I think we should remove the exclusion of 
> {{org/apache/hadoop/thirdparty/**/*}} from the shading config and instead add 
> that pattern to the allowlist so that hbase-shaded-client is all clients need 
> to get started with hbase.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

so we would require a specific patch release of Hadoop as a minimum? It's
messy, but we've certainly done it before.

On Thu, Feb 17, 2022 at 1:31 PM Andrew Purtell  wrote:

> On Thu, Feb 17, 2022 at 10:50 AM Steve Loughran  wrote:
>
> > j
> > On 2022/02/15 22:27:50 Andrew Purtell wrote:
> >
> > > The primary consideration to my mind is the state of S3A: in what
> > version it can be said to be stable and feature complete. I think 3.3 is
> > the appropriate code line for that criteria but perhaps 3.2 could serve
> as
> > well.
> >
> > 3.3.1. i have not been near 3.2.x for many years except for a change for
> > it to be ok with people disabling dir marker deletion
> > (fs.s3a.marker.retention=keep).
> >
> > 3.3.1 is good, with one little surprise:
> > https://issues.apache.org/jira/browse/HADOOP-17771
> > moving to the s3 client builder API for region support (needed for AWS
> > AccessPoint VPN access) broke things unless you explicitly set a region,
> > were running in EC2 *or had the AWS CLI installed*. All tests running in
> > EC2 and all us developers with the CLI installed missed that completely.
> > see the jira for the trivial workaround.
> >
> >
> > you need 3.3.1 for up to date abfs connections too.
> >
> > there's a 3.3.2 release due real soon now...testing the latest RC against
> > HBase would be wonderful.
> >
> > -steve
> >
>
> Thank you for writing in Steve. I think by the time we bring this in for a
> landing over here 3.3.2 will be out, that gets my vote.
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
>- A23, Welcome, Apocalypse
>

[jira] [Created] (HBASE-26760) LICENSE handling should not allow non-aggregated "apache-2.0"

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26760:
---

 Summary: LICENSE handling should not allow non-aggregated 
"apache-2.0"
 Key: HBASE-26760
 URL: https://issues.apache.org/jira/browse/HBASE-26760
 Project: HBase
  Issue Type: Task
  Components: community
Affects Versions: 2.4.3, 2.4.10
Reporter: Sean Busbey
 Fix For: 2.5.0


our non-aggregated license check currently allows the malformed name 
"apache-2.0". All ALv2 licenses should be aggregated.

- remove "apache-2.0" from the {{non_aggregate_fine}} list
- figure out what dependencies are referring to this license and correct them 
in {{supplemental-models.xml}}

nice to have: maybe add a precommit check that complains if the 
{{non_aggregate_fine}} set is changed?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26696) Migrate our jenkins jobs from ci-hadoop to ci-hbase

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26696.
-
Resolution: Fixed

this is all wrapped up now.

> Migrate our jenkins jobs from ci-hadoop to ci-hbase
> ---
>
> Key: HBASE-26696
> URL: https://issues.apache.org/jira/browse/HBASE-26696
> Project: HBase
>  Issue Type: Task
>  Components: jenkins
>Reporter: Duo Zhang
>Priority: Major
>
> We have our own jenkins controller now.
> Let's do the migration.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26756) remove disabled hbase jobs from ci-hadoop

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26756.
-
Resolution: Fixed

went ahead and cleared things out.

> remove disabled hbase jobs from ci-hadoop
> -
>
> Key: HBASE-26756
> URL: https://issues.apache.org/jira/browse/HBASE-26756
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>
> There are a bunch of disabled jobs on ci-hadoop from us cautiously moving to 
> ci-hbase. Delete them all once we're confident that things are moved over to 
> ci-hbase enough that we are going to handle future problems by fixing them 
> in-place on ci-hbase rather than rolling back to running on ci-hadoop.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26758) HBase Thirdparty repo should have nightly for checking main repo tests on top of SNAPSHOT artifacts

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26758:
---

 Summary: HBase Thirdparty repo should have nightly for checking 
main repo tests on top of SNAPSHOT artifacts
 Key: HBASE-26758
 URL: https://issues.apache.org/jira/browse/HBASE-26758
 Project: HBase
  Issue Type: Task
  Components: integration tests, thirdparty
Reporter: Sean Busbey


We should have a nightly test that runs if there are changes to

* top level pom
* any of the hbase-shaded-* modules

That test should grab the current HEAD of a configurable branch of the main 
repo (which defaults to the repo's default branch, currently master) and then 
attempt a build + tests while setting {{hbase-thirdparty.version}} to the 
in-progress SNAPSHOT build. The test run should exclude the current set of 
flaky tests.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

Unfortunately if we want to keep jdk8 working we can't rely on building
with the jdk release target option on newer jdks.

We ran into this recently in HBASE-25465 where a JDK bug came up.

On Tue, Feb 15, 2022 at 8:12 PM 张铎(Duo Zhang)  wrote:

> +1 on moving the minimum required java version for compiling HBase to Java
> 11. But we could still use --release=8 to still support jdk8 at runtime.
>
> Java 8 is still widely used, and I believe this will last for even more
> years, as lots of big companies such as RedHat, Microsoft and Amazon are
> still shipping new jdk8 releases, I do not think it is time to fully drop
> the support of jdk8.
>
> Thanks.
>
> Sean Busbey  于2022年2月16日周三 09:57写道：
>
> > If we set the minJDK to 11 I believe that will effectively remove jdk8
> > support, rather than "just" deprecate it.
> >
> > As we build out more robust testing I would be in favor of running less
> of
> > it on deprecated JDKs.
> >
> >
> > On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:
> >
> > > Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable
> to
> > > me.
> > >
> > > Gotta start pushing the issue somehow.
> > >
> > > On 2/15/22 1:47 PM, Sean Busbey wrote:
> > > > Hi folks!
> > > >
> > > > It's been some time since we decided to stick to LTS JDK releases as
> a
> > > way
> > > > of getting a handle on the JDK treadmill.
> > > >
> > > > What do folks think about deprecating JDK8? The openjdk8u project is
> > > still
> > > > going and there are commercial support options at least through 2030.
> > > >
> > > > Deprecating it in HBase 3 would mean we could remove it in HBase 4,
> not
> > > > that we would _have_ to remove it. The way I think about likely
> timing
> > of
> > > > these events goes like this:
> > > >
> > > > * HBase 2 started alphas in June 2017, betas in January 2018, and
> came
> > > out
> > > > in April 2018
> > > > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > > > discussed how close we are to our stated beta goals (upgrades from
> > active
> > > > 2.x releases and removal of not-ready features).
> > > >
> > > > Given the above, in the absence of us specifically pushing to roll
> > > through
> > > > major version numbers for some reason, I think a reasonably
> > conservative
> > > > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> > > HBase
> > > > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> > > >
> > > > That all said, our current reference guide section on java versions
> > does
> > > > not sound very confident about JDK11 support.
> > > >
> > > >> A Note on JDK11 *
> > > >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > > > support is limited to
> > > >> compilation and running the full test suite. There are open
> questions
> > > > regarding the runtime
> > > >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > > > (HADOOP-15338).
> > > >> Significantly, neither project has yet released a version with
> > explicit
> > > > runtime support for
> > > >> JDK11. The remaining known issues in HBase are catalogued in
> > > HBASE-22972.
> > > >>
> > > >
> > > > Since that blurb was written, Hadoop has added JDK11 support [1] as
> has
> > > > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could
> > update
> > > > our minimum supported versions of these projects to match that
> support.
> > > >
> > > > What do folks think?
> > > >
> > > > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > > > [2]:
> > > >
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> > > >
> > >
> >
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

> Regarding the original question, I would be in favor of the proposal. Time
> marches on. I assume just to state the obvious that our destination of
> minimum LTS would shift from 8 to 11.


Yes, sorry I should have expressly stated JDK11 would become the minimum
with some release after HBase 3.

I got here because I wanted to start working on qualifying JDK17 as a
runtime environment but then realized we were putting more caveats on JDK11
than I expected.

Hadoop 2 isn’t exactly dead, at least the source branch is still receiving
> occasional update, but is not releasing. We should probably consider it
> effectively EOL.


IIRC we've already dropped Hadoop 2 support for HBase 3.

The Hadoop minimum could become 3.3. The primary consideration to my mind
> is the state of S3A: in what version it can be said to be stable and
> feature complete. I think 3.3 is the appropriate code line for that
> criteria but perhaps 3.2 could serve as well.


I really like this as a criteria. Anyone else have an idea on this?



On Tue, Feb 15, 2022, 16:28 Andrew Purtell  wrote:

> The section in our docs on JDK11 seems out of date. In my experience using
> Java 11 to run ZooKeeper (3.5), Hadoop HDFS/YARN/MR (2.10 and 3.3), and
> HBase (2.4) is a nonissue. These software versions are stable under load.
> Most recently I tested scale ingest with 11.0.15+1, hot off the presses so
> to speak. No issues to report — although, I cannot claim this is the
> production configuration where I work yet. Perhaps near the end of 2022.
> Anyway, I think mine is likely not the only experience like this so the
> hedging language can be removed from the text, especially if someone else
> writes in with positive testimonial.
>
> Regarding the original question, I would be in favor of the proposal. Time
> matches on. I assume just to state the obvious that our destination of
> minimum LTS would shift from 8 to 11.
>
> I also agree it would be good to move up from EOL or soon to be EOL
> versions of our dependencies. Looks like ZooKeeper 3.6 will be the lowest
> live code line by the end of this year. Hadoop 2 isn’t exactly dead, at
> least the source branch is still receiving occasional update, but is not
> releasing. We should probably consider it effectively EOL. The Hadoop
> minimum could become 3.3. The primary consideration to my mind is the state
> of S3A: in what version it can be said to be stable and feature complete. I
> think 3.3 is the appropriate code line for that criteria but perhaps 3.2
> could serve as well.
>
> > On Feb 15, 2022, at 10:48 AM, Sean Busbey  wrote:
> >
> > Hi folks!
> >
> > It's been some time since we decided to stick to LTS JDK releases as a
> way
> > of getting a handle on the JDK treadmill.
> >
> > What do folks think about deprecating JDK8? The openjdk8u project is
> still
> > going and there are commercial support options at least through 2030.
> >
> > Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> > that we would _have_ to remove it. The way I think about likely timing of
> > these events goes like this:
> >
> > * HBase 2 started alphas in June 2017, betas in January 2018, and came
> out
> > in April 2018
> > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > discussed how close we are to our stated beta goals (upgrades from active
> > 2.x releases and removal of not-ready features).
> >
> > Given the above, in the absence of us specifically pushing to roll
> through
> > major version numbers for some reason, I think a reasonably conservative
> > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> HBase
> > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> >
> > That all said, our current reference guide section on java versions does
> > not sound very confident about JDK11 support.
> >
> >> A Note on JDK11 *
> >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > support is limited to
> >> compilation and running the full test suite. There are open questions
> > regarding the runtime
> >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > (HADOOP-15338).
> >> Significantly, neither project has yet released a version with explicit
> > runtime support for
> >> JDK11. The remaining known issues in HBase are catalogued in
> HBASE-22972.
> >>
> >
> > Since that blurb was written, Hadoop has added JDK11 support [1] as has
> > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
> > our minimum supported versions of these projects to match that support.
> >
> > What do folks think?
> >
> > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > [2]:
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
>

Re: [DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

If we set the minJDK to 11 I believe that will effectively remove jdk8
support, rather than "just" deprecate it.

As we build out more robust testing I would be in favor of running less of
it on deprecated JDKs.


On Tue, Feb 15, 2022, 16:10 Josh Elser  wrote:

> Deprecating jdk8 for HBase 3 and requiring minJdk=11 seems reasonable to
> me.
>
> Gotta start pushing the issue somehow.
>
> On 2/15/22 1:47 PM, Sean Busbey wrote:
> > Hi folks!
> >
> > It's been some time since we decided to stick to LTS JDK releases as a
> way
> > of getting a handle on the JDK treadmill.
> >
> > What do folks think about deprecating JDK8? The openjdk8u project is
> still
> > going and there are commercial support options at least through 2030.
> >
> > Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
> > that we would _have_ to remove it. The way I think about likely timing of
> > these events goes like this:
> >
> > * HBase 2 started alphas in June 2017, betas in January 2018, and came
> out
> > in April 2018
> > * HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
> > discussed how close we are to our stated beta goals (upgrades from active
> > 2.x releases and removal of not-ready features).
> >
> > Given the above, in the absence of us specifically pushing to roll
> through
> > major version numbers for some reason, I think a reasonably conservative
> > estimate is for HBase 3 to arrive in late 2022 or early 2023 and then
> HBase
> > 4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.
> >
> > That all said, our current reference guide section on java versions does
> > not sound very confident about JDK11 support.
> >
> >> A Note on JDK11 *
> >> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
> > support is limited to
> >> compilation and running the full test suite. There are open questions
> > regarding the runtime
> >> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
> > (HADOOP-15338).
> >> Significantly, neither project has yet released a version with explicit
> > runtime support for
> >> JDK11. The remaining known issues in HBase are catalogued in
> HBASE-22972.
> >>
> >
> > Since that blurb was written, Hadoop has added JDK11 support [1] as has
> > ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
> > our minimum supported versions of these projects to match that support.
> >
> > What do folks think?
> >
> > [1]: https://hadoop.apache.org/docs/r3.3.0/index.html
> > [2]:
> > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq
> >
>

[DISCUSS] deprecating jdk8, progress on LTS jdk support

[Sean Busbey]

Hi folks!

It's been some time since we decided to stick to LTS JDK releases as a way
of getting a handle on the JDK treadmill.

What do folks think about deprecating JDK8? The openjdk8u project is still
going and there are commercial support options at least through 2030.

Deprecating it in HBase 3 would mean we could remove it in HBase 4, not
that we would _have_ to remove it. The way I think about likely timing of
these events goes like this:

* HBase 2 started alphas in June 2017, betas in January 2018, and came out
in April 2018
* HBase 3 started alphas in July 2021, and as of Feb 2022 we haven't
discussed how close we are to our stated beta goals (upgrades from active
2.x releases and removal of not-ready features).

Given the above, in the absence of us specifically pushing to roll through
major version numbers for some reason, I think a reasonably conservative
estimate is for HBase 3 to arrive in late 2022 or early 2023 and then HBase
4 to start alphas in ~2025. An HBase 5 prior to 2030 seems unlikely.

That all said, our current reference guide section on java versions does
not sound very confident about JDK11 support.

> A Note on JDK11 *
> Preliminary support for JDK11 is introduced with HBase 2.3.0. This
support is limited to
> compilation and running the full test suite. There are open questions
regarding the runtime
> compatibility of JDK11 with Apache ZooKeeper and Apache Hadoop
(HADOOP-15338).
> Significantly, neither project has yet released a version with explicit
runtime support for
> JDK11. The remaining known issues in HBase are catalogued in HBASE-22972.
>

Since that blurb was written, Hadoop has added JDK11 support [1] as has
ZooKeeper[2]. As a part of buttoning up our JDK11 support we could update
our minimum supported versions of these projects to match that support.

What do folks think?

[1]: https://hadoop.apache.org/docs/r3.3.0/index.html
[2]:
https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_systemReq

[jira] [Created] (HBASE-26756) remove disabled hbase jobs from ci-hadoop

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26756:
---

 Summary: remove disabled hbase jobs from ci-hadoop
 Key: HBASE-26756
 URL: https://issues.apache.org/jira/browse/HBASE-26756
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey


There are a bunch of disabled jobs on ci-hadoop from us cautiously moving to 
ci-hbase. Delete them all once we're confident that things are moved over to 
ci-hbase enough that we are going to handle future problems by fixing them 
in-place on ci-hbase rather than rolling back to running on ci-hadoop.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26740) migrate ARM specific build

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26740.
-
Resolution: Fixed

I changed the job's label from {{arm2}} to {{arm}} after moving it to ci-hbase. 
job is running now. Will check back on the results tomorrow.

> migrate ARM specific build
> --
>
> Key: HBASE-26740
> URL: https://issues.apache.org/jira/browse/HBASE-26740
> Project: HBase
>  Issue Type: Sub-task
>        Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26755) HBase website generation should show error details on failure

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26755:
---

 Summary: HBase website generation should show error details on 
failure
 Key: HBASE-26755
 URL: https://issues.apache.org/jira/browse/HBASE-26755
 Project: HBase
  Issue Type: Bug
  Components: website
Reporter: Sean Busbey
Assignee: Sean Busbey


website build failed this morning while running maven. no details because we 
have to opt-in via {{--errors}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26743) migrate hbase-thirdparty PR

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26743.
-
Resolution: Fixed

> migrate hbase-thirdparty PR 
> 
>
> Key: HBASE-26743
> URL: https://issues.apache.org/jira/browse/HBASE-26743
> Project: HBase
>  Issue Type: Sub-task
>  Components: jenkins
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26743) migrate hbase-thirdparty PR

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26743:
---

 Summary: migrate hbase-thirdparty PR 
 Key: HBASE-26743
 URL: https://issues.apache.org/jira/browse/HBASE-26743
 Project: HBase
  Issue Type: Sub-task
  Components: jenkins
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26740) migrate ARM specific build

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26740:
---

 Summary: migrate ARM specific build
 Key: HBASE-26740
 URL: https://issues.apache.org/jira/browse/HBASE-26740
 Project: HBase
  Issue Type: Sub-task
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26737) migrate hbase-native-client PR job

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26737.
-
Resolution: Fixed

looks good. got a failure and a success w/proper notification to github.

> migrate hbase-native-client PR job
> --
>
> Key: HBASE-26737
> URL: https://issues.apache.org/jira/browse/HBASE-26737
> Project: HBase
>  Issue Type: Sub-task
>  Components: jenkins
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26737) migrate hbase-native-client PR job

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26737:
---

 Summary: migrate hbase-native-client PR job
 Key: HBASE-26737
 URL: https://issues.apache.org/jira/browse/HBASE-26737
 Project: HBase
  Issue Type: Sub-task
  Components: jenkins
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26736) migrate hhbase-operator-tools PR job

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26736.
-
Resolution: Fixed

> migrate hhbase-operator-tools PR job
> 
>
> Key: HBASE-26736
> URL: https://issues.apache.org/jira/browse/HBASE-26736
> Project: HBase
>  Issue Type: Sub-task
>  Components: jenkins
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26736) migrate hhbase-operator-tools PR job

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26736:
---

 Summary: migrate hhbase-operator-tools PR job
 Key: HBASE-26736
 URL: https://issues.apache.org/jira/browse/HBASE-26736
 Project: HBase
  Issue Type: Sub-task
  Components: jenkins
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26735) migrate hbase-connectors PR job

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26735.
-
Resolution: Fixed

> migrate hbase-connectors PR job
> ---
>
> Key: HBASE-26735
> URL: https://issues.apache.org/jira/browse/HBASE-26735
> Project: HBase
>  Issue Type: Sub-task
>  Components: jenkins
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26735) migrate hbase-connectors PR job

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26735:
---

 Summary: migrate hbase-connectors PR job
 Key: HBASE-26735
 URL: https://issues.apache.org/jira/browse/HBASE-26735
 Project: HBase
  Issue Type: Sub-task
  Components: jenkins
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-26698) migrate website generation

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26698.
-
Resolution: Fixed

> migrate website generation
> --
>
> Key: HBASE-26698
> URL: https://issues.apache.org/jira/browse/HBASE-26698
> Project: HBase
>  Issue Type: Sub-task
>  Components: jenkins, website
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (HBASE-24936) review Jenkins build artifacts

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-24936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-24936.
-
  Assignee: Duo Zhang  (was: Sean Busbey)
Resolution: Fixed

> review Jenkins build artifacts
> --
>
> Key: HBASE-24936
> URL: https://issues.apache.org/jira/browse/HBASE-24936
> Project: HBase
>  Issue Type: Task
>        Reporter: Sean Busbey
>Assignee: Duo Zhang
>Priority: Critical
> Attachments: usage-hbase-find-flaky.zip, usage-hbase-full.zip
>
>
> Post move to the ci-hadoop build servers we are now the biggest user of 
> space. That is not a problem in and of itself, but the master node has run 
> out of disk space twice now. As of this snapshot we are using 125GB of 
> storage and the next largest project is only using 20GB.
> https://paste.apache.org/kyrds
> We should review our builds for any issues and come up with expectations for 
> what our steady-state disk usage should look like
> * we are supposed to compress any test logs (usually this gets us 90-99% 
> space savings)
> * we are supposed to clean up workspaces when jobs are done
> * we are supposed to keep a fixed window of prior builds (either by days or 
> number of runs)
> If all of our jobs are currently following these guidelines, another 
> possibility is to push the artifacts we need over to 
> [nightlies.a.o|https://nightlies.apache.org/authoring.html]. Barring that, we 
> should formally request asf infra set up [a plugin for storing artifact on 
> s3|https://plugins.jenkins.io/artifact-manager-s3/].



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26704) EOL HBase 2.3.x

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26704:
---

 Summary: EOL HBase 2.3.x
 Key: HBASE-26704
 URL: https://issues.apache.org/jira/browse/HBASE-26704
 Project: HBase
  Issue Type: Task
  Components: community
Reporter: Sean Busbey


VOTE thread is here:

https://lists.apache.org/thread/v3gfoyf255rpvql4h3g68ong5p4ps4rs

* remove from https://hbase.apache.org/downloads
* delete from dist svn repo https://dist.apache.org/repos/dist/release/hbase/ 
(which will remove the release from downloads.apache.org)
* send an announcement to user@hbase and dev@hbase



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [VOTE] EOL for HBase 2.3

[Sean Busbey]

Here's my +1.

That's 6 in favor and no other votes. I'll file an issue to track the clean
up work

On Sat, Jan 22, 2022 at 11:25 AM Peter Somogyi  wrote:

> +1
>
> On Sat, Jan 22, 2022 at 6:22 PM Nick Dimiduk  wrote:
>
> > +1
> >
> > On Fri, Jan 21, 2022 at 09:58 Sean Busbey  wrote:
> >
> > > Hi folks!
> > >
> > > Discussion threads:
> > >
> > > https://s.apache.org/i56se
> > > https://s.apache.org/w1rit
> > >
> > > tl;dr: we look to have consensus but missed formalizing things.
> > >
> > > +1 for EOL?
> > >
> >
>

Re: New dedicated Jenkins Conrtoller

[Sean Busbey]

It looks like I currently do have access to manage the ci-hbase controller.


Could you check that the git-websites label handling where a common machine
gets leased to the ci-hbase controller is set up properly? or give me a
pointer on how I would check this myself?

I started trying to migrate our website generation job again and it's been
waiting for a node with the git-websites label to show up for ~8 hours at
this point.

https://ci-hbase.apache.org/job/hbase_generate_website/3/



On Sat, Jan 22, 2022 at 2:42 AM Gavin McDonald  wrote:

> Hi,
> Timestamper plugin installed.
>
> By the way I thought I had given the PMC members permissions to manage
> Jenkins
> on ci-hbase including installing plugins etc, can someone check?
>
>
> On Sat, Jan 22, 2022 at 7:34 AM 张铎(Duo Zhang) 
> wrote:
>
> > We faced a strange issue..
> >
> > rg.codehaus.groovy.control.MultipleCompilationErrorsException: startup
> failed:
> > WorkflowScript: 29: Invalid option type "timestamps". Valid option
> types: [buildDiscarder, catchError, checkoutToSubdirectory,
> disableConcurrentBuilds, disableResume, durabilityHint,
> githubProjectProperty, newContainerPerStage, overrideIndexTriggers,
> parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds,
> retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout,
> waitUntil, warnError, withChecks, withContext, withCredentials, withEnv,
> wrap, ws] @ line 29, column 5.
> >timestamps()
> >^
> >
> > 1 error
> >
> >   at
> org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
> >   at
> org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
> >   at
> org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
> >   at
> org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
> >   at
> org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
> >   at
> groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
> >   at
> groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
> >   at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
> >   at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> >   at
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
> >   at
> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
> >   at
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:571)
> >   at
> org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:523)
> >   at
> org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:334)
> >   at
> hudson.model.ResourceController.execute(ResourceController.java:97)
> >   at hudson.model.Executor.run(Executor.java:429)
> >
> >
> > After googling, it seems a restart of jenkins can fix it, or we do not
> > have timestamper plugin installed? Mind taking a look Gavin?
> >
> > Thanks.
> >
> > 张铎(Duo Zhang)  于2022年1月22日周六 11:40写道：
> >
> >> Oh, just saw that we could do this by ourselves.
> >>
> >> Let me have a try.
> >>
> >> 张铎(Duo Zhang)  于2022年1月22日周六 11:18写道：
> >>
> >>> OK, so while copying you could keep the old jenkins builds?
> >>> That's awesome.
> >>>
> >>> Then please help 'copying' HBase-Find-Flaky-Tests and HBase-Flaky-Tests
> >>> to ci-hbase too? We need the history builds to calucate the flaky test
> list
> >>> so if all the old build data can be kept then it will help us a lot.
> >>>
> >>> Thanks~
> >>>
> >>> 张铎(Duo Zhang)  于2022年1月22日周六 11:03写道：
> >>>
> >>>> Thank you Gavin.
> >>>>
> >>>> Let me first move some light weighted job first to see if everything
> is
> >>>> OK.
> >>>>
> >>>> HBase Nightly, some flaky related jobs, and also the main pre commit
> >>>> job for HBase are tied together. We need to move them at once.
> >>>>
> >>>> Gavin McDonald  于2022年1月22日周六 02:44写道：
> >>>>
> >>>>> Thanks for your patience
> >>>>>
> >>>>> I have 'copied' over a job -
> >>>>> https://ci-hbase.apache.org/job/HBase%20Nightly/
> >>>>>
> >>>>> which of course made me aware of what plugins I ne

Re: New dedicated Jenkins Conrtoller

[Sean Busbey]

Gavin please install the Timestamper plugin

https://plugins.jenkins.io/timestamper/

On Fri, Jan 21, 2022 at 9:40 PM 张铎(Duo Zhang)  wrote:

> Oh, just saw that we could do this by ourselves.
>
> Let me have a try.
>
> 张铎(Duo Zhang)  于2022年1月22日周六 11:18写道：
>
>> OK, so while copying you could keep the old jenkins builds?
>> That's awesome.
>>
>> Then please help 'copying' HBase-Find-Flaky-Tests and HBase-Flaky-Tests
>> to ci-hbase too? We need the history builds to calucate the flaky test list
>> so if all the old build data can be kept then it will help us a lot.
>>
>> Thanks~
>>
>> 张铎(Duo Zhang)  于2022年1月22日周六 11:03写道：
>>
>>> Thank you Gavin.
>>>
>>> Let me first move some light weighted job first to see if everything is
>>> OK.
>>>
>>> HBase Nightly, some flaky related jobs, and also the main pre commit job
>>> for HBase are tied together. We need to move them at once.
>>>
>>> Gavin McDonald  于2022年1月22日周六 02:44写道：
>>>
>>>> Thanks for your patience
>>>>
>>>> I have 'copied' over a job -
>>>> https://ci-hbase.apache.org/job/HBase%20Nightly/
>>>>
>>>> which of course made me aware of what plugins I needed to install.
>>>>
>>>> Feel free to delete that, or use it and delete the old, and let me know
>>>> if have any issues
>>>> with any other missing plugins.
>>>>
>>>>
>>>>
>>>> On Thu, Jan 20, 2022 at 1:11 PM 张铎(Duo Zhang) 
>>>> wrote:
>>>>
>>>>> I’ve also replied on the jira issue.
>>>>>
>>>>> You can try to copy a job to the new Jenkins.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Gavin McDonald 于2022年1月20日 周四18:45写道：
>>>>>
>>>>>> Im taking a look ...
>>>>>>
>>>>>> On Thu, Jan 20, 2022 at 6:54 AM Sean Busbey 
>>>>>> wrote:
>>>>>>
>>>>>> > Hi Gavin
>>>>>> >
>>>>>> > I went to start migrating jobs to the ci-hbase controller, but the
>>>>>> new job
>>>>>> > workflow doesn't include pipelines or multibranch pipelines. We use
>>>>>> both of
>>>>>> > these job types for nearly all of our work. Can they be enabled?
>>>>>> >
>>>>>> > On Tue, Jan 18, 2022 at 5:04 AM Gavin McDonald <
>>>>>> gmcdon...@apache.org>
>>>>>> > wrote:
>>>>>> >
>>>>>> >>
>>>>>> >>
>>>>>> >> On 2022/01/18 09:20:24 "张铎(Duo Zhang)" wrote:
>>>>>> >> > Thank you Gavin.
>>>>>> >> >
>>>>>> >> > We have some urls in our jenkins files which references the
>>>>>> artifacts of
>>>>>> >> > the jenkins jobs, so a simple copy or move can not work. For
>>>>>> example,
>>>>>> >> here:
>>>>>> >> >
>>>>>> >> >
>>>>>> >>
>>>>>> https://github.com/apache/hbase/blob/c9bcd87b34a15d200a55ec7fdc2b1d86e3367a8c/dev-support/Jenkinsfile_GitHub#L47
>>>>>> >> >
>>>>>> >> > So I prefer we migrate the jobs by ourselves. Will file a HBase
>>>>>> issue
>>>>>> >> to do
>>>>>> >> > this.
>>>>>> >> >
>>>>>> >> > And will report any problems on INFRA_22760 while migrating.
>>>>>> When the
>>>>>> >> > migration is half done, you can move for example, 5 hbase nodes
>>>>>> from
>>>>>> >> > ci-hadoop to ci-hbase, and once the migration is finally done,
>>>>>> you can
>>>>>> >> move
>>>>>> >> > all the remaining hbase nodes from ci-hadoop to ci-hbase.
>>>>>> >> >
>>>>>> >> > WDYT?
>>>>>> >>
>>>>>> >> However you want to do it is fine with me :)
>>>>>> >>
>>>>>> >> You have 10 nodes on there right now to get you started.
>>>>>> >>
>>>>>> >> Let me know how you get on
>>>>>> >

[jira] [Created] (HBASE-26698) migrate website generation

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26698:
---

 Summary: migrate website generation
 Key: HBASE-26698
 URL: https://issues.apache.org/jira/browse/HBASE-26698
 Project: HBase
  Issue Type: Sub-task
  Components: jenkins, website
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] Replace log4j with reload4j for branch-2.x

[Sean Busbey]

It's relevant to what kind of mitigation this is. The effectiveness of
reload4j to deal with "the critical CVEs of log4j 1" is limited by how
likely it is that they know about them.

Otherwise at the next CVE we're back in the same place where downstream
users aren't meaningfully more protected. And in that case perhaps we would
do better for our users by e.g. putting more emphasis upgrading across our
releases or providing breaking changes to get the pain over with.

On Fri, Jan 21, 2022 at 11:03 AM Andrew Purtell 
wrote:

> That does not seem germane to this discussion. We don’t investigate and
> attempt to manage the security reporting arrangement of any of our other
> third party dependencies.
>
> > On Jan 21, 2022, at 7:59 AM, Sean Busbey  wrote:
> >
> > Has anyone asked the ASF Logging PMC if they'll forward security reports
> > against log4j 1 to the reload4j project?
> >
> >> On Fri, Jan 21, 2022 at 3:33 AM Pankaj Kumar 
> wrote:
> >>
> >> +1 for reload4j.
> >>
> >> Regards,
> >> Pankaj
> >>
> >>> On Fri, Jan 21, 2022, 2:39 PM 张铎(Duo Zhang) 
> wrote:
> >>>
> >>> Already filed HBASE-26691.
> >>>
> >>> Wei-Chiu Chuang  于2022年1月21日周五 16:53写道：
> >>>
> >>>> +1 I am doing the same in Hadoop.
> >>>>
> >>>> On Fri, Jan 21, 2022 at 4:51 PM Viraj Jasani 
> >> wrote:
> >>>>
> >>>>> +1 for Reload4J migration in active release branches.
> >>>>>
> >>>>>
> >>>>> On Fri, 21 Jan 2022 at 12:52 PM, Andrew Purtell <
> >>>> andrew.purt...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> +1 for migrating to Reload4J. It is binary and configuration
> >>> compatible
> >>>>>> with log4j 1 so meets our compatibility guidelines.
> >>>>>>
> >>>>>> If this is an agreeable plan I can make the changes in a PR and we
> >>> can
> >>>> do
> >>>>>> a round of new releases.
> >>>>>>
> >>>>>>> On Jan 20, 2022, at 10:16 PM, Duo Zhang 
> >>> wrote:
> >>>>>>>
> >>>>>>> On master we have already migrated to log4j2, but for all other
> >>>>> release
> >>>>>>> lines we are still on log4j1.
> >>>>>>>
> >>>>>>> Recently there are several new CVEs for log4j1, so I think we
> >>> should
> >>>>> also
> >>>>>>> address them for release lines other than master.
> >>>>>>>
> >>>>>>> One possible solution is to also migrate log4j2 but use log4j12
> >>>> bridge
> >>>>> to
> >>>>>>> maintain the compatibility, but we have already known that
> >> log4j12
> >>>>> bridge
> >>>>>>> can not work perfectly with hadoop, as hadoop has some customized
> >>>>> log4j1
> >>>>>>> appender implementations, which inherit some log4j1 appenders
> >> which
> >>>> are
> >>>>>> not
> >>>>>>> part of the log4j12 bridge.
> >>>>>>>
> >>>>>>> Reload4j is a fork of the log4j1 and has fixed the critical CVEs,
> >>> so
> >>>> it
> >>>>>> is
> >>>>>>> less hurt to replace log4j with reload4j.
> >>>>>>>
> >>>>>>> Suggestions are welcomed.
> >>>>>>>
> >>>>>>> Thanks. Regards
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>

[VOTE] EOL for HBase 2.3

[Sean Busbey]

Hi folks!

Discussion threads:

https://s.apache.org/i56se
https://s.apache.org/w1rit

tl;dr: we look to have consensus but missed formalizing things.

+1 for EOL?

Re: [DISCUSS] status of HBase 2.3.x

[Sean Busbey]

I would rather have a VOTE that makes our consensus clear for things like
EOL for branches, especially one that has had the stable pointer
previously. I don't think lazy consensus is a sufficient bar in that case.

It sounds like everyone is agreeable to EOL so I'll go start the thread.

On Thu, Jan 20, 2022 at 2:00 PM Nick Dimiduk  wrote:

> Sounds like an omission on my part -- I just didn't follow through on the
> necessary steps after our decision to EOL. I'd rather not VOTE because
> there was already lazy consensus around the discussions.
>
> Regardless of remaining internal community action, is there a checklist for
> RMs to follow when a release line reaches end of life?
>
> Thanks,
> Nick
>
> On Thu, Jan 20, 2022 at 9:17 AM Andrew Purtell 
> wrote:
>
> > Let’s vote. The stable pointer has been moved so there should be no
> vetos.
> >
> > > On Jan 20, 2022, at 7:25 AM, Sean Busbey  wrote:
> > >
> > > Hi folks!
> > >
> > > Our Peter was asking in Slack about how to find the announcement for
> EOL
> > on
> > > HBase 2.3 releases. I did some digging around and I think there's been
> a
> > > miscommunication.
> > >
> > > The page header on our project's space on the ASF downloads page[1]
> > > currently says 2.3 was EOL in October 2021. But none of the usual
> cleanup
> > > has happened; namely 2.3.7 is still in our project download hosting and
> > > listed on our download page.
> > >
> > > I’ve found this discussion of EOL 2.3 which we listed in our board
> > report:
> > > https://s.apache.org/i56se
> > >
> > > That discussion basically ended with "we need to have the stable
> pointer
> > > updated to 2.4 first". After that discussion 2.3.7 was released. I
> can't
> > > find any further discussion or a VOTE thread.
> > >
> > > Personally I think we ought to just do a quick VOTE to make it EOL and
> > > clean things up.
> > >
> > > What do others think?
> > >
> > > [1]: https://downloads.apache.org/hbase/
> >
>

Re: [DISCUSS] Replace log4j with reload4j for branch-2.x

[Sean Busbey]

Has anyone asked the ASF Logging PMC if they'll forward security reports
against log4j 1 to the reload4j project?

On Fri, Jan 21, 2022 at 3:33 AM Pankaj Kumar  wrote:

> +1 for reload4j.
>
> Regards,
> Pankaj
>
> On Fri, Jan 21, 2022, 2:39 PM 张铎(Duo Zhang)  wrote:
>
> > Already filed HBASE-26691.
> >
> > Wei-Chiu Chuang  于2022年1月21日周五 16:53写道：
> >
> > > +1 I am doing the same in Hadoop.
> > >
> > > On Fri, Jan 21, 2022 at 4:51 PM Viraj Jasani 
> wrote:
> > >
> > > > +1 for Reload4J migration in active release branches.
> > > >
> > > >
> > > > On Fri, 21 Jan 2022 at 12:52 PM, Andrew Purtell <
> > > andrew.purt...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 for migrating to Reload4J. It is binary and configuration
> > compatible
> > > > > with log4j 1 so meets our compatibility guidelines.
> > > > >
> > > > > If this is an agreeable plan I can make the changes in a PR and we
> > can
> > > do
> > > > > a round of new releases.
> > > > >
> > > > > > On Jan 20, 2022, at 10:16 PM, Duo Zhang 
> > wrote:
> > > > > >
> > > > > > On master we have already migrated to log4j2, but for all other
> > > > release
> > > > > > lines we are still on log4j1.
> > > > > >
> > > > > > Recently there are several new CVEs for log4j1, so I think we
> > should
> > > > also
> > > > > > address them for release lines other than master.
> > > > > >
> > > > > > One possible solution is to also migrate log4j2 but use log4j12
> > > bridge
> > > > to
> > > > > > maintain the compatibility, but we have already known that
> log4j12
> > > > bridge
> > > > > > can not work perfectly with hadoop, as hadoop has some customized
> > > > log4j1
> > > > > > appender implementations, which inherit some log4j1 appenders
> which
> > > are
> > > > > not
> > > > > > part of the log4j12 bridge.
> > > > > >
> > > > > > Reload4j is a fork of the log4j1 and has fixed the critical CVEs,
> > so
> > > it
> > > > > is
> > > > > > less hurt to replace log4j with reload4j.
> > > > > >
> > > > > > Suggestions are welcomed.
> > > > > >
> > > > > > Thanks. Regards
> > > > >
> > > >
> > >
> >
>

[DISCUSS] status of HBase 2.3.x

[Sean Busbey]

Hi folks!

Our Peter was asking in Slack about how to find the announcement for EOL on
HBase 2.3 releases. I did some digging around and I think there's been a
miscommunication.

The page header on our project's space on the ASF downloads page[1]
currently says 2.3 was EOL in October 2021. But none of the usual cleanup
has happened; namely 2.3.7 is still in our project download hosting and
listed on our download page.

I’ve found this discussion of EOL 2.3 which we listed in our board report:
https://s.apache.org/i56se

That discussion basically ended with "we need to have the stable pointer
updated to 2.4 first". After that discussion 2.3.7 was released. I can't
find any further discussion or a VOTE thread.

Personally I think we ought to just do a quick VOTE to make it EOL and
clean things up.

What do others think?

[1]: https://downloads.apache.org/hbase/

Re: New dedicated Jenkins Conrtoller

[Sean Busbey]

Hi Gavin

I went to start migrating jobs to the ci-hbase controller, but the new job
workflow doesn't include pipelines or multibranch pipelines. We use both of
these job types for nearly all of our work. Can they be enabled?

On Tue, Jan 18, 2022 at 5:04 AM Gavin McDonald  wrote:

>
>
> On 2022/01/18 09:20:24 "张铎(Duo Zhang)" wrote:
> > Thank you Gavin.
> >
> > We have some urls in our jenkins files which references the artifacts of
> > the jenkins jobs, so a simple copy or move can not work. For example,
> here:
> >
> >
> https://github.com/apache/hbase/blob/c9bcd87b34a15d200a55ec7fdc2b1d86e3367a8c/dev-support/Jenkinsfile_GitHub#L47
> >
> > So I prefer we migrate the jobs by ourselves. Will file a HBase issue to
> do
> > this.
> >
> > And will report any problems on INFRA_22760 while migrating. When the
> > migration is half done, you can move for example, 5 hbase nodes from
> > ci-hadoop to ci-hbase, and once the migration is finally done, you can
> move
> > all the remaining hbase nodes from ci-hadoop to ci-hbase.
> >
> > WDYT?
>
> However you want to do it is fine with me :)
>
> You have 10 nodes on there right now to get you started.
>
> Let me know how you get on
>
> >
> > Thanks.
> >
> > Gavin McDonald  于2022年1月18日周二 16:16写道：
> >
> > > Hi All,
> > >
> > > I created an Infra ticket for tracking.
> > >
> > > https://issues.apache.org/jira/browse/INFRA-22760
> > >
> > > For now, I left your current nodes hbase1 to hbase10 in ci-hadoop.a.o
> > >
> > > I have created for you 10 new nodes so far out of our Hetzner DC
> > > of a similar specifications to the hbase1-10.
> > >
> > > Those are hbase11-20 and are available right now for testing on
> > > ci-hbase.apache.org
> > >
> > > I do not know your workflow well enough to decide the next steps for
> you
> > > for testing.
> > >
> > > Options include:
> > >
> > > 1. I can 'move' all jobs out of ci-hadoop onto ci-hbase
> > > 2. I can 'move' one or a number of jobs out of ci-hadoop onto ci-hbase
> > > 3. Similar to 1 or 2, but 'copy' any number of jobs over for testing.
> > > - I like copy, however quite a few projects jobs would not be
> > > appropriate
> > >   to have the same job running twice, including those that deploy,
> or
> > > those that
> > >   build from PRs and continue a workflow to report back, or approve
> > > for merge.
> > >   So for a copy, would need to be a job that just 'builds and
> tests'
> > > with no outflow.
> > > 4. You guys create new jobs on ci-hbase.apache.org to test the nodes
> > > setup and the
> > > main controller setup, without affecting current jobs.
> > >
> > > Obviously, you may to tweak version controlled script files such as a
> > > JenkinsFile, which makes the copy options harder unless testing from a
> > > branch created for the purpose.
> > >
> > > I may be overthinking things, and option 1 , the simplest and most
> > > straight forward from my end would also suit you guys and we deal with
> any
> > > breakage as it happens (I'm thinking of integrations with other
> services,
> > > Credentials/Secrets, Plugins I haven't installed that are needed etc.)
> > >
> > > Let me know please ASAP how you want to proceed so we can get using
> these
> > > nodes, after which I can then proceed to move hbase1-10 nodes over
> also.
> > > And then start adding more into the mix.
> > >
> > > Oh. The 'git-websites' label should be a no-op. It is available on
> > > ci-hbase.a.o as the nodes with that label are shared and leased out to
> > > controllers as needed upon request of a job.
> > >
> > > Thanks
> > >
> > > Gav...
> > >
> > >
> > > On 2021/12/09 15:07:26 "张铎(Duo Zhang)" wrote:
> > > > Thanks you Gavin.
> > > >
> > > > You can do it in your convenience, just let me know so I could
> change the
> > > > label in the jenkinsfile of our jenkins jobs, at least our pre
> commit job
> > > > still uses the Hadoop label.
> > > >
> > > > And another thing is about the hbase_generate_website job, it needs
> to be
> > > > executed on the nodes under the git-websites label. I'm not sure if
> the
> > > > label is still available after the migration.
> > > >
> > > > Gavin McDonald  于2021年12月8日周三 17:28写道：
> > > >
> > > > > Hi HBase devs.
> > > > >
> > > > > This is to let you know that Infra has created a dedicated Jenkins
> > > > > Controller at https://ci-hbase.apache.org .
> > > > >
> > > > > You have new incoming donated agents which will get connected to
> this
> > > new
> > > > > controller by Infra.
> > > > >
> > > > > You also have 10 or so agents connected to
> > > https://ci-hadoop.apache.org -
> > > > > I
> > > > > would like to move these agents - and all of your jobs - over to
> the
> > > new
> > > > > controller as soon as possible.
> > > > >
> > > > > You may or may not need to reconfigure your jobs and or
> JenkinsFile or
> > > > > other.
> > > > >
> > > > > Can I get the go ahead to move these at my convenience or suggest
> > > please a
> > > > > timeline for Infra to move these agents.
> > > > >
> > > > 

Re: A tweak to our checkstyle configuration

[Sean Busbey]

I don't feel strongly about the specific proposal that started this thread.
I could see why someone might find it easier to skim the code; I don't feel
it impacts my reading personally. If there are folks with differing
opinions over it; my preference would be that we stick with whatever style
is already more prevalent in the codebase.

I do feel strongly that we need _some_ consistently enforced standards. I
also think we've been paying a sort of tax as a community for a long time
because our existing code base does not currently follow our stated
standards, so it's confusing. Our section on code conventions[1] doesn't
really give any background beyond "common feedback" as to how we came to
our current formatting rules, but it does tell folks that the relevant
starting point is the sun java conventions[2].

IIRC our current style guidelines grew out of our time in the Apache Hadoop
project which also follows the sun java conventions except with 2 spaces
per indent level instead of 4[3]. It's been modified a bit over time as
things didn't work for us. the biggest difference that comes to mind is our
longer max line length. and maybe the import ordering?

I don't think we should adopt some alternate coding standard all at once
without a good summary of how it compares to our current formatting
standards.

I would be in favor of paying the one-time cost of bringing our active
release lines into compliance with our stated code format standards. This
could be done, for example, by module to break up the reviewing.

[1]: https://hbase.apache.org/book.html#common.patch.feedback
[2]:
https://www.oracle.com/java/technologies/javase/codeconventions-contents.html
[3]:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute#HowToContribute-MakingChanges



On Sat, Jan 15, 2022 at 3:07 AM 张铎(Duo Zhang)  wrote:

> What about just switching to use google java style?
>
> Nick Dimiduk  于2022年1月13日周四 03:22写道：
>
> > Hey all.
> >
> > Discussion on the PR has resulted in an impasse of opinion, but also
> > renewed interest in improvements to static analysis in general
> > (HBASE-26617).
> >
> > I think that this kind of code hygiene is very important for the
> long-term
> > maintenance of a large project like ours and especially one that accepts
> > contributions from a broad audience. I would really appreciate it if some
> > more folks would chime into these discussions on PRs, or bring your
> > concerns back up to this thread. I'm game to help see the work done, but
> we
> > need more voices to participate in defining what is required by the
> > community.
> >
> > Thanks in advance,
> > Nick
> >
> > On Thu, Dec 9, 2021 at 3:58 PM Nick Dimiduk  wrote:
> >
> > > Heya,
> > >
> > > I have posted a small change to our checkstyle configuration on
> > > HBASE-26536. This change will relax the whitespace rules regarding the
> > > left-curly-bracket ('{') character. Specifically, I intend this change
> to
> > > allow short expressions that include a nested scope that fits entirely
> on
> > > one line. The example I provide is:
> > >
> > > if (foo == null) { return null; }
> > >
> > > This whitespace style is already present (though I think not in popular
> > > usage) within the codebase. Please take a look and let me know if you
> > have
> > > any concerns about making this change.
> > >
> > > Thanks,
> > > Nick
> > >
> > > https://issues.apache.org/jira/browse/HBASE-26536
> > > https://github.com/apache/hbase/pull/3913
> > >
> >
>

Re: [DISCUSS] Keep EOL HBase and Hadoop versions in our support matrix for longer time

[Sean Busbey]

We already publish version specific reference guides as a part of our
binary tarballs. My understanding is that's a big part of why they're
still in the main source repository; that we get docs built as a part
of our assembly.

e.g. just picking a 2.4 release I have handy:

(base) sbusbey@seans-mbp ~ % tar tzf Downloads/hbase-2.4.8-bin.tar.gz|
grep pdf
hbase-2.4.8/docs/apache_hbase_reference_guide.pdf
(base) sbusbey@seans-mbp ~ % tar tzf Downloads/hbase-2.4.8-bin.tar.gz|
grep book.html
hbase-2.4.8/docs/book.html

Historically these do get maintained for each minor version. My
understanding is that the diligence on backporting across both
contributors to docs and release managers has varied considerably over
time.

If we're only going to have one version of the docs, then we should
break the docs out of the main repo entirely so that we can simplify
their generation.

IIRC we have only gone through the trouble of publishing to the
website version specific API docs / ref guides for release lines that
made it to be the "stable" branch.

On Sat, Jan 8, 2022 at 9:37 AM 张铎(Duo Zhang)  wrote:
>
> I agree with Andrew that maybe it is beyond our ability to maintain
> ref guides for different release lines and keep them all in sync...
>
> What about this, we just make the ref guide for the master branch in
> sync, which contains all release lines. We will still remove
> information of the EOL releases as needed, to keep the ref guide
> clean, but as said in the title, less aggressive.
> And when we decide to EOL a release line, we copy the ref guide of the
> current master branch to the specific branch, and generate the ref
> guide for that release line for the last time.
> In this way the release manager does not need to always think of
> keeping the ref guide in sync, we all just need to consider the master
> one, only one extra work needs to be done when EOLing a release line.
>
> WDYT?
>
> Thanks.
>
> Andrew Purtell  于2022年1月8日周六 07:16写道：
> >
> > There are some challenges with respect to keeping multiple versions of the
> > documentation around. Each minor release needs a new version? Each RM
> > managing one or more code line(s) needs to update trunk docs and also
> > backport to branch docs (or not, depending)? There's been a JIRA open
> > forever for a 2.4 version of the book and honestly I haven't found time to
> > do it, because it seems both low priority and nontrivial, and there's never
> > enough time for everything... although that may be a personal failing.
> >
> > On Fri, Jan 7, 2022 at 9:05 AM Sean Busbey  wrote:
> >
> > > We should have a version specific version of the ref guide that
> > > contains that information.
> > >
> > > e.g.
> > >
> > > https://hbase.apache.org/1.4/book.html#hadoop
> > >
> > > https://hbase.apache.org/2.3/book.html#hadoop
> > >
> > > Can we do a better job of making these discoverable to folks rather
> > > than keeping stuff around?
> > >
> > > On Fri, Jan 7, 2022 at 1:14 AM 张铎(Duo Zhang) 
> > > wrote:
> > > >
> > > > Recently we've seen several emails on the user list asking whether
> > > > some hbase versions support specific hadoop versions, usually it will
> > > > be some versions which are already EOL, so there is no information in
> > > > our ref guide.
> > > >
> > > > For me I think we could leave the EOL versions in the support matrix
> > > > for a bit longer. It will be useful for our users.
> > > >
> > > > Thanks.
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk

Re: [DISCUSS] Keep EOL HBase and Hadoop versions in our support matrix for longer time

[Sean Busbey]

We should have a version specific version of the ref guide that
contains that information.

e.g.

https://hbase.apache.org/1.4/book.html#hadoop

https://hbase.apache.org/2.3/book.html#hadoop

Can we do a better job of making these discoverable to folks rather
than keeping stuff around?

On Fri, Jan 7, 2022 at 1:14 AM 张铎(Duo Zhang)  wrote:
>
> Recently we've seen several emails on the user list asking whether
> some hbase versions support specific hadoop versions, usually it will
> be some versions which are already EOL, so there is no information in
> our ref guide.
>
> For me I think we could leave the EOL versions in the support matrix
> for a bit longer. It will be useful for our users.
>
> Thanks.

Re: [DISCUSS] In flight work to complete before 2.5.0RC0

[Sean Busbey]

If we don't want to wait for HBASE-26543 (fix arg parsing for shell)
then we should revert HBASE-24772 from branch-2.5 prior to an RC.


On Wed, Dec 8, 2021 at 7:34 PM Andrew Purtell  wrote:
>
> As your branch-2.5 RM I am assembling a list of work items that should be
> completed before a 2.5.0RC0 candidate is submitted for the PMC's
> consideration.
>
> I have so far:
>
> - OpenTracing span naming convention and coverage improvements.
>
> - Shell exit code fixes/improvements.
>
> - The "encryption improvements umbrella". Arguable, but let's include it
> for now. Can all be resolved as Later if need be.
>
> Let's discuss what else, if anything, should be on this list, or if one or
> more of the above items does not constitute a release blocker. I consider
> incomplete work-in-progress a blocker. Obviously all of the work in
> progress should land before release. For WIP, let's also agree on a
> definition of done.
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk

Re: [DISCUSS] releasing hbase-connector 1.1.0 ?

[Sean Busbey]

That'd be great!

I can help with release process.

I'd like us to ensure this go around we have convenience binaries compiled
against  Spark 3 as well. That's been a long standing need.

On Mon, Dec 6, 2021, 16:43 Tak Lon (Stephen) Wu  wrote:

> Hi guys,
>
> While I'm learning how to release a minor version of hbase via
> https://hbase.apache.org/book.html#releasing and an example of 2.3.7
> via HBASE-26373, I found hbase-connector was last released on 2019
> April.
>
> So, I'm wondering if I could help to release hbase-connector e.g.
> 1.1.0 and get familiar with the release process, at the same time, I
> may need a mentor/adviser and discuss the steps/tasks about it (my
> plan is to create JIRAs like HBASE-26373).
>
> any comments?
>
> Thanks,
> Stephen
>

[jira] [Created] (HBASE-26543) HBase shell no longer properly parses several args

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26543:
---

 Summary: HBase shell no longer properly parses several args
 Key: HBASE-26543
 URL: https://issues.apache.org/jira/browse/HBASE-26543
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 3.0.0-alpha-1, 2.5.0
Reporter: Sean Busbey
Assignee: Sean Busbey
 Fix For: 2.5.0, 3.0.0-alpha-2


shell cli arg parsing is broken:

* debug is eating a possible arg
* -D handling is consuming part of ARGV without declaring it
* non-interactive mode is eating a possible arg
* top level definitions is eating a possible arg
* script2run is being ignored entirely




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (HBASE-26469) HBase shell has changed exit behavior

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26469:
---

 Summary: HBase shell has changed exit behavior
 Key: HBASE-26469
 URL: https://issues.apache.org/jira/browse/HBASE-26469
 Project: HBase
  Issue Type: Bug
  Components: shell
Affects Versions: 2.4.8, 2.5.0, 3.0.0-alpha-2
Reporter: Sean Busbey


The HBase shell has changed behavior in a way that breaks being able to exit 
properly.

Two example scripts to act as stand ins for hbase shell scripts to "do 
something simple then exit":
{code}
tmp % echo "list\nexit" > clean_exit.rb
tmp % echo "list\nexit 1" > error_exit.rb
{code}

Giving these two scripts is possible:

* passed as a cli argument
* via redirected stdin

Additionally the shell invocation can be:

* in the default compatibility mode
* with the "non interactive" flag that gives an exit code that reflects runtime 
errors

I'll post logs of the details as attachments but here are some tables of the 
exit codes.

The {{clean_exit.rb}} invocations ought to exit with success, exit code 0.

|| ||  1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 || 
master ||
| cli, default |0 |0   |0   |0   |0   |0   |1   |   
 1*   |
| cli, -n | 0 |0   |0   |0   |0   |0   |1   |  
hang   |
| stdin, default |  0 |0   |0   |0   |0   |0   |0   |   
 0|
| stdin, -n |   1 |1   |1   |1   |1   |1   |1*  |   
 1*   |

The {{error_exit.rb}} invocation should return a non-zero exit code, unless 
we're specifically trying to match a normal hbase shell session.

|| || 1.4.14 || 1.7.1 || 2.0.6 || 2.1.9 || 2.2.7 || 2.3.7 || 2.4.8 || 
master ||
| cli, default |   1 |1   |1   |1   |1   |1   |1*  |
1*   |
| cli, -n |1 |1   |1   |1   |1   |1   |1*  |  
hang   |
| stdin, default | 0 |0   |0   |0   |0   |0   |0   |
0|
| stdin, -n |  1 |1   |1   |1   |1   |1   |1*  |
1*   |

In cases marked with * the error details are different.

The biggest concern are the new-to-2.4 non-zero exit code when we should have a 
success and the hanging.

The former looks like this:

{code}
ERROR NoMethodError: private method `exit' called for nil:NilClass
{code}

The change in error details for the error exit script also shows this same 
detail.


This behavior appears to be a side effect of HBASE-11686. As far as I can tell, 
the IRB handling of 'exit' calls fail because we implement our own handling of 
sessoins rather than rely on the intended session interface. We never set a 
current session, and IRB's exit implementation presumes there will be one.

Running in debug shows this in a stacktrace:
{code}
Took 0.4563 seconds
ERROR NoMethodError: private method `exit' called for nil:NilClass
NoMethodError: private method `exit' called for nil:NilClass
 irb_exit at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/extend-command.rb:30
 evaluate at stdin:2
 eval at org/jruby/RubyKernel.java:1048
 evaluate at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/workspace.rb:85
  eval_io at uri:classloader:/shell.rb:327
 each_top_level_statement at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:246
 loop at org/jruby/RubyKernel.java:1442
 each_top_level_statement at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:232
catch at org/jruby/RubyKernel.java:1189
 each_top_level_statement at 
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/irb/ruby-lex.rb:231
  eval_io at uri:classloader:/shell.rb:326
  classpath:/jar-bootstrap.rb at classpath:/jar-bootstrap.rb:194
exception_handler at uri:classloader:/shell.rb:339
at classpath:/jar-bootstrap.rb:194
{code}

And in our version of IRB (0.9.6) [line 30 for 
extend-commands|https://github.com/ruby/irb/blob/v0.9.6/lib/irb/extend-command.rb#L30]
 corresponds to this code:
{code}
# Quits the current irb context
#
# +ret+ is the optional signal or message to send to Context#exit
#
# Same as IRB.CurrentContext.exit.
def irb_exit(ret = 0)
  irb_context.exit(ret)
end
{code}




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [DISCUSS] Introducing OpenTelemerty into our public API

[Sean Busbey]

If it was akin to slf4j, that integration has been relatively smooth
for downstream.

log4j effectively being in our public api through configuration
formats has been a maintenance nightmare.

These would effectively be in our java binary API though, right? Would
we gain any meaningful isolation from trouble later on if we had
wrapper objects around the otel stuff?

On Tue, Nov 9, 2021 at 2:29 PM Nick Dimiduk  wrote:
>
> Heya,
>
> I've been kicking the tires on the OpenTelemetry tracing work and I have an
> ugly question to present. One of the things suggested by the otel community
> is that library implementers (like us, with hbase-client) expose the
> Span/Scope objects to callers so that they can add context annotations. I
> think this means adding API calls that return otel instances. From our
> previous experience exposing Guava objects in our API, this sounds no good.
> On the other hand, log4j and SLF4j are effectively in our public API, and
> so maybe otel is of a similar breed.
>
> I have not yet attempted to implement exposure of their objects in our
> client API, so as of now, my position is that of speculation. I'll report
> back as I make progress in this direction. If anyone else has already
> traveled this road, please speak up and share your experiences.
>
> Thanks,
> Nick

Re: [DISCUSS] JIRA Version page only available to authenticated users

[Sean Busbey]

for example, this is roughly what HBase 2.4.7 would look like:

https://gist.github.com/busbey/b5a849626bc4fcc2cbdd18f56c8d2807#file-apache-hbase-2-4-7-releasenotes-md

(but the url would be to the rel/2.4.7 tag on github)

On Tue, Oct 26, 2021 at 1:33 PM Sean Busbey  wrote:

> let's just start pointing at the release notes? for any given release that
> should be where we as a community have spent the most effort crafting a
> message for our downstream folks.
>
> preferably rendered. Could we start putting the generated release note
> that just the part relevant to the specific patch release into the GitHub
> release tag's release note? IIRC that will render markdown and is readable
> without logging in. We could integrate populating it via the GitHub API
> into the release tooling.
>
> On Tue, Oct 26, 2021 at 11:39 AM Nick Dimiduk  wrote:
>
>> I find it surprising that our announcements point people at something that
>> requires a login. It seems not very “open” or “inviting”.
>>
>> On Tue, Oct 26, 2021 at 07:26 Sean Busbey  wrote:
>>
>> > We have historically linked to the Jira Version page because it
>> provides a
>> > starting point on issue navigation that we don't get elsewhere. Plenty
>> of
>> > interactions with our issue tracker require a login. Why is that
>> threshold
>> > not acceptable here? I think we should just note that a login with
>> > low-friction signup is required. Though I guess we will need to validate
>> > that it's still straightforward to create a jira account.
>> >
>> > The changelog and release notes for the release are both available on
>> > downloads.a.o if we want to link to something available without a login
>> > rather than just mentioning them. If we're going to change what we'd
>> link
>> > to I'd rather one of those than jira at all.
>> >
>> > On Tue, Oct 26, 2021 at 5:55 AM 张铎(Duo Zhang) 
>> > wrote:
>> >
>> > > At least for me I haven't typed it up in the past...
>> > >
>> > > I'm not sure whether you need to set up a template manually first...
>> > >
>> > > Wei-Chiu Chuang  于2021年10月26日周二
>> 下午5:48写道：
>> > >
>> > > > Duo,
>> > > > Just out of curiosity, is this release note in the JIRA populated
>> > > > automatically? Or someone typed it up?
>> > > >
>> > > > On Tue, Oct 26, 2021 at 4:25 PM 张铎(Duo Zhang) <
>> palomino...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > I think this is what we should link in the announcement?
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753=12350545
>> > > > >
>> > > > > We should link the release note, not the page where we can
>> generate
>> > the
>> > > > > release note...
>> > > > >
>> > > > > Just my thoughts. Thanks.
>> > > > >
>> > > > > Nick Dimiduk  于2021年10月26日周二 上午1:19写道：
>> > > > >
>> > > > > > Heya,
>> > > > > >
>> > > > > > The link we include in our announcement emails is only
>> accessible
>> > for
>> > > > > > people who are logged into JIRA. Seems like bad form.
>> > > > > >
>> > > > > > I.e.,
>> > > https://issues.apache.org/jira/projects/HBASE/versions/12350545
>> > > > > >
>> > > > > > I doubt this is by design on our part. I wonder if JIRA has
>> always
>> > > been
>> > > > > > configured this way, or if it's an (un)intentional change on the
>> > part
>> > > > of
>> > > > > > Infra.
>> > > > > >
>> > > > > > Should we instead link to something else?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Nick
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] JIRA Version page only available to authenticated users

[Sean Busbey]

let's just start pointing at the release notes? for any given release that
should be where we as a community have spent the most effort crafting a
message for our downstream folks.

preferably rendered. Could we start putting the generated release note that
just the part relevant to the specific patch release into the GitHub
release tag's release note? IIRC that will render markdown and is readable
without logging in. We could integrate populating it via the GitHub API
into the release tooling.

On Tue, Oct 26, 2021 at 11:39 AM Nick Dimiduk  wrote:

> I find it surprising that our announcements point people at something that
> requires a login. It seems not very “open” or “inviting”.
>
> On Tue, Oct 26, 2021 at 07:26 Sean Busbey  wrote:
>
> > We have historically linked to the Jira Version page because it provides
> a
> > starting point on issue navigation that we don't get elsewhere. Plenty of
> > interactions with our issue tracker require a login. Why is that
> threshold
> > not acceptable here? I think we should just note that a login with
> > low-friction signup is required. Though I guess we will need to validate
> > that it's still straightforward to create a jira account.
> >
> > The changelog and release notes for the release are both available on
> > downloads.a.o if we want to link to something available without a login
> > rather than just mentioning them. If we're going to change what we'd link
> > to I'd rather one of those than jira at all.
> >
> > On Tue, Oct 26, 2021 at 5:55 AM 张铎(Duo Zhang) 
> > wrote:
> >
> > > At least for me I haven't typed it up in the past...
> > >
> > > I'm not sure whether you need to set up a template manually first...
> > >
> > > Wei-Chiu Chuang  于2021年10月26日周二
> 下午5:48写道：
> > >
> > > > Duo,
> > > > Just out of curiosity, is this release note in the JIRA populated
> > > > automatically? Or someone typed it up?
> > > >
> > > > On Tue, Oct 26, 2021 at 4:25 PM 张铎(Duo Zhang)  >
> > > > wrote:
> > > >
> > > > > I think this is what we should link in the announcement?
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753=12350545
> > > > >
> > > > > We should link the release note, not the page where we can generate
> > the
> > > > > release note...
> > > > >
> > > > > Just my thoughts. Thanks.
> > > > >
> > > > > Nick Dimiduk  于2021年10月26日周二 上午1:19写道：
> > > > >
> > > > > > Heya,
> > > > > >
> > > > > > The link we include in our announcement emails is only accessible
> > for
> > > > > > people who are logged into JIRA. Seems like bad form.
> > > > > >
> > > > > > I.e.,
> > > https://issues.apache.org/jira/projects/HBASE/versions/12350545
> > > > > >
> > > > > > I doubt this is by design on our part. I wonder if JIRA has
> always
> > > been
> > > > > > configured this way, or if it's an (un)intentional change on the
> > part
> > > > of
> > > > > > Infra.
> > > > > >
> > > > > > Should we instead link to something else?
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] JIRA Version page only available to authenticated users

[Sean Busbey]

We have historically linked to the Jira Version page because it provides a
starting point on issue navigation that we don't get elsewhere. Plenty of
interactions with our issue tracker require a login. Why is that threshold
not acceptable here? I think we should just note that a login with
low-friction signup is required. Though I guess we will need to validate
that it's still straightforward to create a jira account.

The changelog and release notes for the release are both available on
downloads.a.o if we want to link to something available without a login
rather than just mentioning them. If we're going to change what we'd link
to I'd rather one of those than jira at all.

On Tue, Oct 26, 2021 at 5:55 AM 张铎(Duo Zhang)  wrote:

> At least for me I haven't typed it up in the past...
>
> I'm not sure whether you need to set up a template manually first...
>
> Wei-Chiu Chuang  于2021年10月26日周二 下午5:48写道：
>
> > Duo,
> > Just out of curiosity, is this release note in the JIRA populated
> > automatically? Or someone typed it up?
> >
> > On Tue, Oct 26, 2021 at 4:25 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > I think this is what we should link in the announcement?
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753=12350545
> > >
> > > We should link the release note, not the page where we can generate the
> > > release note...
> > >
> > > Just my thoughts. Thanks.
> > >
> > > Nick Dimiduk  于2021年10月26日周二 上午1:19写道：
> > >
> > > > Heya,
> > > >
> > > > The link we include in our announcement emails is only accessible for
> > > > people who are logged into JIRA. Seems like bad form.
> > > >
> > > > I.e.,
> https://issues.apache.org/jira/projects/HBASE/versions/12350545
> > > >
> > > > I doubt this is by design on our part. I wonder if JIRA has always
> been
> > > > configured this way, or if it's an (un)intentional change on the part
> > of
> > > > Infra.
> > > >
> > > > Should we instead link to something else?
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > >
> >
>

Re: The nightly jobs for branch-1.x are failing

[Sean Busbey]

we'd need to have a discussion about pushing to docker hub, esp what
our intentions are (i.e. publishing for downstream vs internal use)
and how we communicate that via docker hub. The mechanisms for
publishing to docker hub are relatively straightforward.


On Sun, Oct 10, 2021 at 10:35 PM 张铎(Duo Zhang)  wrote:
>
> Thank you Sean.
>
> Maybe we could do this on branch-1. I think there are still some folks who
> are interested in maintaining new branch-1 releases.
>
> And for branch-1.4, I prefer we do a final release and then mark it as EOL.
>
> And back to the docker build issue, is it possible to push the docker image
> to docker hub so we do not need to build it every time when the cache
> expires?
>
> I saw there is an 'apache' user on docker hub...
>
> https://hub.docker.com/u/apache
>
> Thanks.
>
> Sean Busbey  于2021年10月11日周一 上午4:01写道：
>
> > it could be that the curl in the docker image for branch-1 is older and is
> > missing some newer CAs.
> >
> > Could we sidestep this issue by installing tooling from spotbugs instead?
> >
> > On Fri, Oct 8, 2021 at 10:54 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > They all fail with docker image building failure.
> > >
> > > 10:50:04  Step 19/32 : RUN mkdir -p /opt/findbugs && curl -L -s -S
> > > >
> > > >
> > >
> > https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download
> > > >  -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz
> > > > --strip-components 1 -C /opt/findbugs
> > > > 10:50:05   ---> Running in d3ff60cd19b4
> > > > 10:50:05   [91mcurl: (60) SSL certific [0m [91mate problem: certificate
> > > > has expired
> > > > 10:50:05  More details here: http://curl.haxx.se/docs/sslcerts.html
> > > > 10:50:05
> > > > 10:50:05  curl performs SSL certificate verification by default, using
> > a
> > > > "bundle"
> > > > 10:50:05   of Certificate Authority (CA) public keys (CA certs). If the
> > > > default
> > > > 10:50:05   bundle file isn't adequate, you can specify [0m [91man
> > > > alternate file
> > > > 10:50:05   using the --cacert option.
> > > > 10:50:05  If this HTTPS server uses a certificate signed by a CA
> > > > represented in
> > > > 10:50:05   the bundle, the certificate verification probably [0m [91m
> > > > failed due to a
> > > > 10:50:05   problem with the certificate (it might be expired, or the
> > name
> > > > might
> > > > 10:50:05   not match the domain name in the U [0m [91mRL).
> > > > 10:50:05  If you'd like to turn off curl's verification of the
> > > > certificate, use
> > > > 10:50:05   the -k (or --insecure) opt [0m [91mion.
> > > > 10:50:06   [0mThe command '/bin/sh -c mkdir -p /opt/findbugs &&
> >  curl
> > > > -L -s -S
> > > >
> > >
> > https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download
> > > >  -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz
> > > > --strip-components 1 -C /opt/findbugs' returned a non-zero code: 60
> > > > 10:50:06  ERROR: Docker failed to build yetus/hbase:633d966bfe.
> > > >
> > >
> > > But I tried the curl command locally and it could download the tarball
> > > successfully.
> > >
> > > Not sure what the problem is, did anyone face this problem before?
> > >
> > > Thanks.
> > >
> >

Re: [hbase-connectors] - release to include Spark 3

[Sean Busbey]

Hi Jordan!

What build system are you using? from that snippet I think it's sbt?

The most expedient change on our end would be to publish a spark 3
compatible version with a maven classifier to distinguish it from the
spark 2 version. Before proceeding with that, do you know off hand if
your build tooling can consume maven dependencies with a classifier?

On Fri, Oct 8, 2021 at 3:53 PM Jordan Hambleton
 wrote:
>
> Hi Sean,
>
> Thanks for the reply on this! We are looking for building spark applications 
> with the dependency such as below. I've seen builds packaging deps with 
> different flavors, ie. provided, assembly, shaded as well. Scala version's 
> built against has typically been with the latest v2.12.x releases.
>
> Publishing it to maven central would work.
>
> basic example (note this will fail w/ v1.0.0) -
>
> libraryDependencies ++= Seq(
>   "org.apache.spark" %% "spark-core" % "3.1.2",
>   "org.apache.spark" %% "spark-sql" % "3.1.2",
>   "org.apache.hbase.connectors.spark" % "hbase-spark" % "1.0.0"
> )
>
> Appreciate the help in making it easier to use the hadoop-connectors for 
> Spark 3. Keep me posted if there's any additional information needed.
>
> regards,
> Jordan
>
> On 2021/10/08 16:09:34, Sean Busbey  wrote:
> > Hi Jordan!
> >
> > How do you currently pull in the dependency? Do you need us to publish an
> > artifact to maven central? Would a convenience binary built against spark-3
> > on downloads.apache.org suffice?
> >
> > On Thu, Oct 7, 2021 at 7:55 PM Jordan Hambleton
> >  wrote:
> >
> > > Hi Peter,
> > >
> > > We're seeing an uptick in usage with Spark 3. While hadoop-connectors have
> > > started supporting Spark 3 (HBASE-25326
> > > <https://issues.apache.org/jira/browse/HBASE-25326>) and while it's fine
> > > pulling down the repo and building from source, it would be a lot easier 
> > > if
> > > we had a release for supporting Spark 3.
> > >
> > > Is it possible to get a next release of the hadoop-connectors out or is
> > > there another way others are using for Spark3 integration with HBase?
> > >
> > > Below is what I'm referring to in building.
> > >
> > > To build the connector with Spark 3.0, compile it with scala 2.12.
> > > Additional configurations that you can customize are the Spark version,
> > > HBase version, and Hadoop version. Example:
> > >
> > > $ mvn -Dspark.version=3.0.1 -Dscala.version=2.12.10
> > > -Dscala.binary.version=2.12 -Dhbase.version=2.2.4 -Dhadoop.profile=3.0
> > > -Dhadoop-three.version=3.2.0 -DskipTests clean package
> > >
> > > best regards,
> > > Jordan Hambleton
> > >
> >

[DISCUSS] move stable pointer to 2.4.6+

[Sean Busbey]

Hi folks!

I'd like us to move the stable pointer from 2.3.z releases to 2.4.6+ releases.

The last time we talked about doing so[1] there was a desire to get
some documentation together on what we as a community expect from the
"stable" release line. We have an issue tracking those needs[2], but
AFAICT we haven't had sufficient community interest to get criteria
together over the last 6 months.

The 2.3 release line started in July 2020 and there is active
discussion about declaring it EOL[3]. 2.4 releases have been going
since Dec 2020 and there is attestation that 2.4 has been as or more
stable than 2.3 in a testing environment[4].

Personally, I view the "stable" pointer as simply a way to say "if you
are new to our community we'd like you to use this release." I
personally think 2.4.z currently meets that standard and we need not
block updating on something more rigorous.

What do folks think?

-busbey

[1]:
"[DISCUSS] Updating the 'stable' pointer to 2.4.2" : https://s.apache.org/6cz3t

[2]:
https://issues.apache.org/jira/browse/HBASE-25690

[3]:
"[DISCUSS] EOL 2.3" : https://s.apache.org/pgkge

[4]:
Message on thread "[DISCUSS] EOL 2.3"
https://s.apache.org/ks7wk

Re: The nightly jobs for branch-1.x are failing

[Sean Busbey]

it could be that the curl in the docker image for branch-1 is older and is
missing some newer CAs.

Could we sidestep this issue by installing tooling from spotbugs instead?

On Fri, Oct 8, 2021 at 10:54 PM 张铎(Duo Zhang)  wrote:

> They all fail with docker image building failure.
>
> 10:50:04  Step 19/32 : RUN mkdir -p /opt/findbugs && curl -L -s -S
> >
> >
> https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download
> >  -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz
> > --strip-components 1 -C /opt/findbugs
> > 10:50:05   ---> Running in d3ff60cd19b4
> > 10:50:05   [91mcurl: (60) SSL certific [0m [91mate problem: certificate
> > has expired
> > 10:50:05  More details here: http://curl.haxx.se/docs/sslcerts.html
> > 10:50:05
> > 10:50:05  curl performs SSL certificate verification by default, using a
> > "bundle"
> > 10:50:05   of Certificate Authority (CA) public keys (CA certs). If the
> > default
> > 10:50:05   bundle file isn't adequate, you can specify [0m [91man
> > alternate file
> > 10:50:05   using the --cacert option.
> > 10:50:05  If this HTTPS server uses a certificate signed by a CA
> > represented in
> > 10:50:05   the bundle, the certificate verification probably [0m [91m
> > failed due to a
> > 10:50:05   problem with the certificate (it might be expired, or the name
> > might
> > 10:50:05   not match the domain name in the U [0m [91mRL).
> > 10:50:05  If you'd like to turn off curl's verification of the
> > certificate, use
> > 10:50:05   the -k (or --insecure) opt [0m [91mion.
> > 10:50:06   [0mThe command '/bin/sh -c mkdir -p /opt/findbugs && curl
> > -L -s -S
> >
> https://sourceforge.net/projects/findbugs/files/findbugs/3.0.1/findbugs-noUpdateChecks-3.0.1.tar.gz/download
> >  -o /opt/findbugs.tar.gz && tar xzf /opt/findbugs.tar.gz
> > --strip-components 1 -C /opt/findbugs' returned a non-zero code: 60
> > 10:50:06  ERROR: Docker failed to build yetus/hbase:633d966bfe.
> >
>
> But I tried the curl command locally and it could download the tarball
> successfully.
>
> Not sure what the problem is, did anyone face this problem before?
>
> Thanks.
>

Re: [hbase-connectors] - release to include Spark 3

[Sean Busbey]

Hi Jordan!

How do you currently pull in the dependency? Do you need us to publish an
artifact to maven central? Would a convenience binary built against spark-3
on downloads.apache.org suffice?

On Thu, Oct 7, 2021 at 7:55 PM Jordan Hambleton
 wrote:

> Hi Peter,
>
> We're seeing an uptick in usage with Spark 3. While hadoop-connectors have
> started supporting Spark 3 (HBASE-25326
> ) and while it's fine
> pulling down the repo and building from source, it would be a lot easier if
> we had a release for supporting Spark 3.
>
> Is it possible to get a next release of the hadoop-connectors out or is
> there another way others are using for Spark3 integration with HBase?
>
> Below is what I'm referring to in building.
>
> To build the connector with Spark 3.0, compile it with scala 2.12.
> Additional configurations that you can customize are the Spark version,
> HBase version, and Hadoop version. Example:
>
> $ mvn -Dspark.version=3.0.1 -Dscala.version=2.12.10
> -Dscala.binary.version=2.12 -Dhbase.version=2.2.4 -Dhadoop.profile=3.0
> -Dhadoop-three.version=3.2.0 -DskipTests clean package
>
> best regards,
> Jordan Hambleton
>

Re: [DISCUSS] EOL 2.3

[Sean Busbey]

please start a dedicated DISCUSS thread about moving the stable pointer.

On Thu, Oct 7, 2021 at 11:33 PM Yu Li  wrote:

> Hi all,
>
> Since 2.4.6 has been released on Sep. 13th and we plan to EOL 2.3.x, shall
> we move the stable pointer to 2.4 (it's still on 2.3.6 for now on our
> download page [1])? Or any concerns? Thanks.
>
> Best Regards,
> Yu
>
> [1] https://hbase.apache.org/downloads.html
>
>
> On Tue, 3 Aug 2021 at 06:01, Andrew Purtell 
> wrote:
>
> > I’ve been working with 2.4 for months and it is stable enough for me to
> +1
> > moving the pointer at this time. This work includes many tests with 2.4
> > using ITBLL with slowDeterministic and serverKilling policies and there
> has
> > not been an unstable result, as defined by crashes/loss of daemons by the
> > IT framework, or data issues requiring hbck.
> >
> > > On Aug 2, 2021, at 2:27 PM, Nick Dimiduk  wrote:
> > >
> > > On Mon, Aug 2, 2021 at 1:22 PM Andrew Purtell <
> andrew.purt...@gmail.com
> > >
> > > wrote:
> > >
> > >> EOL of 2.3 seems premature.
> > >>
> > >> The stable pointer is still pointing to 2.3.
> > >>
> > >> Advancing the stable pointer is an obvious prerequisite. If there are
> > any
> > >> concerns blocking advancing the pointer to 2.4, this is where we
> should
> > >> start the discussion.
> > >>
> > >
> > > Point taken. I see the only conclusion of our thread [0] was to agree
> > that
> > > we should define "stable" criteria. I thought we'd already advanced the
> > > marker.
> > >
> > > [0]:
> > >
> >
> https://lists.apache.org/thread.html/r1cc4528a6a35cd1b0d38398aa61ad642a368901795d6970544d0a0a9%40%3Cdev.hbase.apache.org%3E
> > >
> > >>> On Aug 2, 2021, at 11:23 AM, Nick Dimiduk 
> wrote:
> > >>>
> > >>> Heya,
> > >>>
> > >>> I'd like to start a discussion regarding the declaration of the end
> of
> > >> life
> > >>> for branch-2.3. This release line has matured nicely, but with the
> > stable
> > >>> pointer moving forward, and our community interest in pushing on 2.5
> > and
> > >>> 3.0, I think it's worth considering whether we want to continue
> putting
> > >>> resources into this release line.
> > >>>
> > >>> I propose we make one final release, 2.3.7, whenever we have enough
> > >> commits
> > >>> to the branch, or the beginning of October, whichever comes first.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>> Thanks,
> > >>> Nick
> > >>
> >
>

[jira] [Created] (HBASE-26332) Update next set of minor releases for Hadoop 3.1 EOM

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26332:
---

 Summary: Update next set of minor releases for Hadoop 3.1 EOM 
 Key: HBASE-26332
 URL: https://issues.apache.org/jira/browse/HBASE-26332
 Project: HBase
  Issue Type: Task
  Components: hadoop3
Affects Versions: 2.5.0, 3.0.0-alpha-2
Reporter: Sean Busbey
 Fix For: 2.5.0, 3.0.0-alpha-2


Looks like Hadoop 3.1 went EOL back in June. I can't find a formal 
announcement, but "[\[VOTE\] Hadoop 3.1.x 
EOL|https://s.apache.org/hadoop-vote-3.1-eol]; passed and HADOOP-17759 removed 
3.1 from their active downloads page and the asf active distribution mirroring.

We should update our minimum testing version to 3.2 and update the hadoop 
dependency matrix to reflect this change.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26331) verify hbase testing infra is up to date wrt H node renaming

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26331:
---

 Summary: verify hbase testing infra is up to date wrt H node 
renaming
 Key: HBASE-26331
 URL: https://issues.apache.org/jira/browse/HBASE-26331
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Sean Busbey


Infra is renaming all the H* nodes.

https://cwiki.apache.org/confluence/display/INFRA/H+node+renaming+Proposal

do a pass through the Jenkins scripts and manually configured jobs to make sure 
we either do not directly name H* nodes or to update such for the rename.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Contribution of a thrift2 python api

[Sean Busbey]

yes, generally a PMC member associated with a specific release should
push the released artifacts to pypi. Someone will need to run down the
details of doing that, but there is precedent of other ASF projects
doing so.

On Tue, Sep 28, 2021 at 10:14 PM Yutong Xiao  wrote:
>
> Contributing to hbase-connectors is ok for me. But there is a question
> about how to release the package. Currently, I upload the package to pypi
> with my own pypi account. Not clear whether this should change if I
> contribute the client to hbase-connectors.
>
> Reid Chan  于2021年8月3日周二 下午10:27写道：
>
> > Brief summaries from this thread:
> >
> > I think you can push to hbase-connectors
> > <https://github.com/apache/hbase-connectors> repo. Just create a new dir
> > hbase-thrift-python, and put your codes under it.
> > Your codes need to be reviewed by good Pythoners in the community.
> >
> > Some follow-up tasks include (but not limited to):
> >
> >- requirements.txt
> >- docs about how to release
> >- some client examples
> >- add some test codes (I'm not sure this part, whether there are similar
> >conventions in python world)
> >- other related information
> >
> >
> > About the *.thrfit files you mentioned, I can't come up a good idea by now.
> > Looks like we still need to create two separate PRs to update both hbase
> > and hbase-connectors repo.
> > However, as *.thrfit files seldom get updated, I feel it should be a ok.
> >
> >
> > --
> > Best Regards,
> > R.C
> >
> >
> > On Tue, Aug 3, 2021 at 10:42 AM Yutong Xiao  wrote:
> >
> > > Hello, any other thoughts about this?
> > >
> > > Yutong Xiao  于2021年7月21日周三 上午11:30写道：
> > >
> > > > Hi. I have removed the personal info in the licenses. For the point
> > 3.2,
> > > > thbase is dependent on the hbase.thrift file. I have involved the
> > > > hbase.thrift file, that thbase used, in the repo. In this case,  repo
> > > > separation will lead to a sync problem between the hbase.thrfit files
> > in
> > > > HBase repo and the connector repo. I am concerned this may make it hard
> > > for
> > > > maintenance. What do you think?
> > > >
> > > > 张铎(Duo Zhang)  于2021年7月17日周六 上午9:39写道：
> > > >
> > > >> One of the difficulties of moving hbase-thrift and hbase-rest out is
> > > >> because we make use of hbase-http in these two modules, at least for
> > > >> setting up the status servlet...
> > > >>
> > > >> Sean Busbey  于2021年7月16日周五 下午11:01写道：
> > > >>
> > > >> > maybe a good fit for the hbase-connectors repo? I know we've talked
> > a
> > > >> > few times about moving the thrift server out there. if we did both
> > > >> > then the compatibility question becomes just the standard
> > > >> > client/server compatibility provided the thrift server only uses our
> > > >> > public java client API.
> > > >> >
> > > >> > On Thu, Jul 15, 2021 at 10:21 PM Yutong Xiao 
> > > >> wrote:
> > > >> > >
> > > >> > > btw for point 2, if allowed I can do that.
> > > >> > > And for point 3.2 it is only a personal idea, the final decision
> > > >> should
> > > >> > be
> > > >> > > made by the community.
> > > >> > > Besides, many of my python user colleagues started using this
> > > library.
> > > >> > > I think many python users have the demand of a good HBase python
> > > >> client.
> > > >> > >
> > > >> > > Yutong Xiao  于2021年7月16日周五 上午11:07写道：
> > > >> > >
> > > >> > > > 1. The license is no problem.
> > > >> > > > 2. This should see if any committer or PMC has interests to do
> > > that.
> > > >> > > > 3. I can be responsible for those documents. About 3.2, as
> > thbase
> > > >> has
> > > >> > been
> > > >> > > > uploaded to Pypi, I think it would be better if it is a new,
> > > >> separate
> > > >> > repo.
> > > >> > > >
> > > >> > > > Wei-Chiu Chuang  于2021年7月6日周二 上午10:22写道：
> > > >> > > >
> > > >> > > >> Hi
> > > >> > > >> than

[DISCUSS] use of Apache Yetus Audience Annotations

[Sean Busbey]

Hi!

Heads up that a discussion has started in Apache Yetus about dropping
the Audience Annotations and associated javadoc tooling due to lack of
community support[1]. The current cutting issue AFAICT is that things
there haven't been updated for the changes in how doclets are handled
in JDK9+.

We still center all of our API scoping on this library. In general it
has been solid and required relatively little investment on our part.

Personally, I think this has worked really well for us so far and we
ought to try to keep the shared resource going. Unfortunately, I don't
currently have the cycles to personally step up in the Yetus project.

What do folks think? Anyone able to help out in Yetus? Should we start
moving to maintain this tooling internal to HBase?

[1]:
https://s.apache.org/ybdl6
"[DISCUSS] Drop JDK8; audience-annotations" from d...@yetus.apache.org

Re: [DISCUSS] InterfaceStability with InterfaceAudience.Public

[Sean Busbey]

Our API is already too big to audit by hand for breakage. The limited
tooling we have for automatically scanning as a part of the release
process[1] only has the ability to cope with a single set of
annotation (i.e. it can do "filter to things that are IA.Public" and
it can't do "filter to things that are IA.Public and IS.Stable"). As a
practical matter I don't think we can reliably meet promises beyond
"everything IA.Public is stable".

That practical limitation is why the current HBase dev docs call out
our difference from yetus javadocs.

I like the idea of a IA.LimitedPrivate experimental as a way to have a
proving ground for APIs we intend to make public but we want a test
out period in a user facing release. It's a relatively low risk way
for us as a community to see if we and our users find the approach
useful compared to how we currently do things (front load API
discussions and then mark things public; if needed deprecate/remove if
it doesn't work out).

On Thu, Sep 2, 2021 at 9:38 AM 张铎(Duo Zhang)  wrote:
>
> I think the current Compatibility Matrix for our IA.Public APIs is already
> complicated enough, so adding IS annotation to the IA.Public APIs will be a
> huge pain for our end users, so I suppose we should not do this.
> And it is a bit strange that, an IA.Public API is also marked as
> IS.Unsable, right? It seems to just tell users do not use it, as it will be
> broken even in a patch release...
> So in general, I think we should change the javadoc for the IS annotation,
> to mention that we do not IS annotation for IA.Public APIs, it should
> always be IS.Stable.
>
> But looking from the developer side, it is a true pain that, seems there is
> no way for us to introduce 'experimental' APIs.
> So maybe we could add a new LP type called experimental, so these APIs
> could be marked IA.LimitedPrivate("Experimental") and we could use the IS
> annotation then.
>
> This could make developers life easier, but I still a bit worry that, will
> end users actually use these 'Experimental' APIs? If no one will use it
> until it becomes IA.Public, then what's the value for doing this...
>
> Just my simple thoughts.
>
> Thanks.
>
> Bryan Beaudreault  于2021年9月1日周三 上午9:41写道：
>
> > Hello devs,
> >
> > A recent discussion came up on slack related to a PR I'm working on which
> > adds a new class annotated with InterfaceAudience.Public. It seems like
> > there's some disagreement in terms of what the
> > current documented expectations are for InterfaceStability in this case,
> > and what expectations we might actually want. Specifically, should we allow
> > annotating IA.Public classes with IS.Evolving or IS.Unstable?
> >
> > Below I quote two conflicting documents, and I'm curious how the group
> > thinks we should reconcile them. Before I do, I just wanted to put out my
> > opinion that it feels like we should have some ability to push new public
> > classes that might evolve; basically beta features that are part of a
> > normal release.
> >
> > In the dev docs (
> > https://hbase.apache.org/book.html#hbase.client.api.surface),
> > there is this quote:
> >
> > IA.Public classes are inherently stable and adhere to our stability
> > guarantees relating to the type of upgrade (major, minor, or patch).
> > IA.LimitedPrivate classes should always be annotated with one of the given
> > InterfaceStability values. If they are not, you should presume they are
> > IS.Unstable.
> > IA.Private classes should be considered implicitly unstable, with no
> > guarantee of stability between releases.
> >
> > On the other hand, the actual javadoc (
> >
> > https://yetus.apache.org/documentation/in-progress/javadocs/org/apache/yetus/audience/InterfaceStability.htm
> > )
> > for InterfaceStability states:
> >
> > All classes that are annotated with InterfaceAudience.Public or
> > InterfaceAudience.LimitedPrivate must have InterfaceStability annotation.
> > Classes that are InterfaceAudience.Private are to be considered unstable
> > unless a different InterfaceStability annotation states otherwise.
> > Incompatible changes must not be made to classes marked as stable.
> >
> > One interpretation is that these are not in conflict, since one should
> > simply put IS.Stable on Public classes. But it seems like another
> > interpretation is that we just must put _any_ IS annotation.
> >
> > Thoughts?
> >

Re: [DISCUSS] Contribution of a thrift2 python api

[Sean Busbey]

maybe a good fit for the hbase-connectors repo? I know we've talked a
few times about moving the thrift server out there. if we did both
then the compatibility question becomes just the standard
client/server compatibility provided the thrift server only uses our
public java client API.

On Thu, Jul 15, 2021 at 10:21 PM Yutong Xiao  wrote:
>
> btw for point 2, if allowed I can do that.
> And for point 3.2 it is only a personal idea, the final decision should be
> made by the community.
> Besides, many of my python user colleagues started using this library.
> I think many python users have the demand of a good HBase python client.
>
> Yutong Xiao  于2021年7月16日周五 上午11:07写道：
>
> > 1. The license is no problem.
> > 2. This should see if any committer or PMC has interests to do that.
> > 3. I can be responsible for those documents. About 3.2, as thbase has been
> > uploaded to Pypi, I think it would be better if it is a new, separate repo.
> >
> > Wei-Chiu Chuang  于2021年7月6日周二 上午10:22写道：
> >
> >> Hi
> >> thanks for your interest in contributing the python api to the HBase
> >> project.
> >>
> >> I quickly check and it doesn't look like there's another active python
> >> HBase thrift client project at this point.
> >> I don't have a demand to use a python thrift hbase client library. If
> >> there
> >> are people who will benefit from this library, then it's a good idea to
> >> make sure the library is well maintained, by having it become part of the
> >> Apache HBase project and that more developers can contribute to it.
> >>
> >> As a hobbyist Python developer I can help review/commit the patch.
> >>
> >> My two cents:
> >> (1) license: the code is ASL 2.0 so it's compatible. The text "Copyright
> >> 2021 Yutong Sean" would need to be removed.
> >> (2) Apache Infra does not manage PyPi. So we (the Apache HBase project
> >> committers/PMC) will have to do that.
> >> I suspect we will have to replicate this PyPi project and add the
> >> interested HBase PMCs who's willing to do the release work.
> >> (3) compatibility matrix: we need to document what versions of HBase
> >> server
> >> is supported.
> >> (3) code:
> >> (3.1) You will need a requirements.txt and preferably specify the versions
> >> of the dependencies.
> >> (3.2) If the community accepts it, should it be part of the HBase main
> >> repo, or a new, separate repo?
> >>
> >>
> >>
> >> On Mon, Jul 5, 2021 at 7:12 PM Yutong Xiao  wrote:
> >>
> >> > Hi,
> >> >
> >> > I used to have a demand to deploy hbase thrift2 service for python
> >> users.
> >> > So that I developed a python clients API supporting python 2.7 and 3.x
> >> for
> >> > hbase thrift2, named thbase  .
> >> Besides
> >> > that, I also added some features to current thrift2 service (HBASE-26025
> >> >  and
> >> > HBASE-26037
> >> > ). I
> >> > deployed them in the prod environment of my company and are compatible
> >> with
> >> > thbase and I will keep maintaining this python API and add new features.
> >> >   I am glad to contribute thbase to the community, but I am not sure if
> >> it
> >> > is possible that such a client could be contributed to the community. So
> >> > that I would like to get some advice about this.
> >> >
> >> > Thanks,
> >> > Yutong Sean
> >> >
> >>
> >

[jira] [Resolved] (HBASE-26031) Validate nightly builds run on new ci workers hbase11-hbase15

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-26031.
-
Resolution: Later

After chatting with Stack about donation internals, these hosts are getting 
decommed for now.

> Validate nightly builds run on new ci workers hbase11-hbase15
> -
>
> Key: HBASE-26031
> URL: https://issues.apache.org/jira/browse/HBASE-26031
> Project: HBase
>  Issue Type: Task
>  Components: test
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
> Attachments: image-2021-06-24-16-14-03-721.png
>
>
> Per slack, asf infra has finished adding in nodes hbase10-hbase15 to 
> ci-hadoop.
> make sure they can run nightly.
> # Set labels for all these node to "hbase-staging"
> # Push a feature branch off of current HEAD that updates the agent labels to 
> use "hbase-staging"
> # trigger a bunch of runs. make sure *something* runs on each of the nodes
> # Set labels for the nodes to "hbase"
> # delete feature branch



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26031) Validate nightly builds run on new ci workers hbase10-hbase15

[Sean Busbey (Jira)]

Sean Busbey created HBASE-26031:
---

 Summary: Validate nightly builds run on new ci workers 
hbase10-hbase15
 Key: HBASE-26031
 URL: https://issues.apache.org/jira/browse/HBASE-26031
 Project: HBase
  Issue Type: Task
  Components: test
Reporter: Sean Busbey
Assignee: Sean Busbey


Per slack, asf infra has finished adding in nodes hbase10-hbase15 to ci-hadoop.

make sure they can run nightly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: HBase JIRA admin requests

[Sean Busbey]

added!

On Thu, May 27, 2021 at 2:13 PM Reid Chan  wrote:
>
> Hi team,
>
> I'm working on the hbase-1.7.0 release, but found I couldn't add a new tag
> for unreleased versions, could someone grant me the access?
>
> Thanks.
>
> ---
> Best Regards,
> R.C

Re: Time for 3.0.0 release

[Sean Busbey]

I'm happy to see us moving forward with hbase 3 release.

If a feature makes it into alpha releases but under evaluation doesn't look
ready for use, what's the plan? Back things out and put it into a feature
branch?

What about making releases out of the master branch until we stabilize the
API by starting beta releases?

What's our goal for upgrades to 3.0? Any 2.y or some specific minimum?
Rolling upgrade?



On Fri, May 21, 2021, 20:25 张铎(Duo Zhang)  wrote:

> Oh, I forgot a big break change, moving to log4j2.
>
> 张铎(Duo Zhang)  于2021年5月21日周五 下午11:10写道：
>
> > Since favored nodes is an existing feature, an improvement for an
> existing
> > feature can come in at a minor release I think, unless you plan to
> > completely break the compatibility.
> >
> > Mallikarjun  于2021年5月21日周五 下午10:11写道：
> >
> >> For multi tenancy with favoured nodes, timeline looks unreasonable for
> >> 3.0.
> >> Can it be part of later 3.x releases? Or should it wait for 4.0?
> >>
> >> On Fri, May 21, 2021, 7:30 PM 张铎(Duo Zhang) 
> >> wrote:
> >>
> >> > We already have the below big feature/changes for 3.0.0.
> >> >
> >> > Synchronous Replication
> >> > OpenTelemetry Tracing
> >> > Distributed MOB Compaction
> >> > Backup and Restore
> >> > Move RSGroup balancer to core
> >> > Reimplement sync client on async client
> >> > CPEPs on shaded proto
> >> >
> >> > There are also some ongoing works which target 3.0.0.
> >> >
> >> > Splittable meta
> >> > Move balancer code to hbase-balancer
> >> > Compaction offload
> >> > Replication offload
> >> >
> >> > Since now, we do not even have enough new features to cut a minor
> >> release
> >> > for 2.x, I think it is time to cut the 3.x release line now, and I
> >> think we
> >> > already have enough new features for a new major release.
> >> >
> >> > Here I plan to cut a branch-3 at the end of June and make our first
> >> > 3.0.0-alpha1 release, and finally make the 3.0.0 release by the end of
> >> > 2021. So if any of the above work can not be done before the end of
> >> June,
> >> > they will be moved out to 4.0.0.
> >> >
> >> > Thoughts? Thanks.
> >> >
> >>
> >
>

Re: [DISCUSS] Next 2.x releases

[Sean Busbey]

My understanding is that backup work is not ready for inclusion in 2.x.

The talk of removing it from the master branch and proposed adoption of the
feature through more involvement from some community members were not so
long ago.


On Tue, May 4, 2021, 15:49 Andrew Purtell  wrote:

> Correct me if I am mistaken but backup tests failing on master in precommit
> is common enough to warrant ignoring them as unrelated. Is it not fully
> baked yet?
>
> +1 for backport of tracing. If we do the backport to branch-2 that would be
> one new compelling reason for a 2.5.0 release.
>
>
> On Tue, May 4, 2021 at 9:34 AM Nick Dimiduk  wrote:
>
> > On Fri, Apr 30, 2021 at 5:40 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > Does anyone have interest in backporting HBASE-22120 to branch-2?
> > >
> >
> > Yes, I think there would be interest in getting your tracing effort onto
> > branch-2 ; there's quite a few "watchers" on that Jira.
> >
> > What about the backup work? Has it stabilized enough for backport?
> >
> > Conversely, if we don't have a killer new feature for 2.5, does that mean
> > it's time for 3.0?
> >
> > Andrew Purtell  于2021年5月1日周六 上午5:46写道：
> > >
> > > > Inline
> > > >
> > > > On Fri, Apr 30, 2021 at 2:11 PM Nick Dimiduk 
> > > wrote:
> > > >
> > > > > Heya,
> > > > >
> > > > > I'd like to have a planning discussion around our 2.x releases. 2.4
> > > seems
> > > > > to be humming along nicely, I think we won't have a need for 2.3
> much
> > > > > longer. Likewise, it seems like it should be time to start planning
> > > 2.5,
> > > > > but when I look at the issues unique to that branch [0], I don't
> see
> > > the
> > > > > big new feature that justifies the new minor release. Rather, I
> see a
> > > > > number of items that should be backported to a 2.4.x. Do we have a
> > big
> > > > new
> > > > > feature in the works? Should we consider backporting something from
> > > > master?
> > > > > Or maybe there's enough minor changes on 2.5 to justify the
> > release...
> > > > but
> > > > > if so, which of them motivate users to upgrade?
> > > > >
> > > > > So, to summarize:
> > > > >
> > > > > - How much longer do we want to run 2.3?
> > > > > - What issues in the below query can be backported to 2.4? Any
> > > > volunteers?
> > > > >
> > > >
> > > > Thanks for starting this discussion, Nick.
> > > >
> > > > Looking at that report, issues like HBASE-24305, HBASE-25793, or
> > > > HBASE-25458 that clean up or reduce interfaces or refactor/move
> classes
> > > are
> > > > probably excluded from a patch release by our guidelines. Conversely,
> > > they
> > > > would not provide much value if backported.
> > > >
> > > > I also agree that the motivation for 2.5 here is thin as of now.
> > > Refactors,
> > > > interface improvements, or deprecation removals will be nice-to-haves
> > > when
> > > > there is a 2.5 someday.
> > > >
> > > > All the others in the report are either operational improvements that
> > > would
> > > > be nice to backport or bug fixes that should be backported.
> > > >
> > > > There might be case by case issues with compatibility during the pick
> > > > backs, but we can deal with them one at a time.
> > > >
> > > > If you are looking for a volunteer to do this, as 2.4 RM I am game.
> > > >
> > > >
> > > > - What's the big new feature that motivates 2.5?
> > > > >
> > > > > Thanks,
> > > > > Nick
> > > > >
> > > > > [0]:
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.5.0%20AND%20fixVersion%20not%20in%20(2.4.0%2C%202.4.1%2C%202.4.2%2C%202.4.3)%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > >- A23, Crosstalk
> > > >
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

[jira] [Resolved] (HBASE-25555) release hbase-thirdparty 3.5.0

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-2.
-
Resolution: Fixed

> release hbase-thirdparty 3.5.0
> --
>
> Key: HBASE-2
> URL: https://issues.apache.org/jira/browse/HBASE-2
> Project: HBase
>  Issue Type: Task
>  Components: hbase-thirdparty
>    Reporter: Sean Busbey
>    Assignee: Sean Busbey
>Priority: Major
> Fix For: hbase-thirdparty-3.5.0
>
>
> make a release for hbase-thirdparty off of the current master branch.
> Since HBASE-24802 adds a new dependency artifact we should increment the 
> minor version number, which gives us a proposed version of 3.5.0.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[ANNOUNCE] Apache HBase Thirdparty 3.5.0 released

[Sean Busbey]

We've just made a new release of the Apache HBase Thirdparty project.
This project is used by the Apache HBase project to encapsulate a
number of core dependencies that HBase relies upon ensuring
that they are properly isolated from HBase downstream users, e.g.
Google Protocol Buffers, Google Guava, and a few others.

The Apache HBase Thirdparty 3.5.0 release includes a drop in replacement
for the defunct HTrace project's library. This artifact is binary and
source compatible with the artifact produced by the HTrace incubating 4.2.0
release with the exception of APIs that expressly referred to the relocated
version of Jackson it included.

See the release notes and change log for more details on fixes included in
this release:

https://downloads.apache.org/hbase/hbase-thirdparty-3.5.0/RELEASENOTES.md

https://downloads.apache.org/hbase/hbase-thirdparty-3.5.0/CHANGES.md

This release is available as source from the ASF mirror infrastructure and
as artifacts in maven central.

- Your Release Manager

[ANNOUNCE] Apache HBase Operator Tools 1.1.0 is now available for download

[Sean Busbey]

The HBase team is happy to announce the immediate availability of Apache
HBase Operator Tools 1.1.0.

Apache HBase™ Operator Tools provides HBCK2 which is the repair tool for
Apache HBase 2 clusters.
To learn more about HBase and HBase Operator Tools, see
https://hbase.apache.org/.

The full list of issues can be found in the included CHANGES.md and
RELEASENOTES.md, or via our issue tracker:

https://s.apache.org/hbase-operator-tools-1.1.0-jira

To download please follow the links and instructions on our website:

https://hbase.apache.org/downloads.html

Question, comments, and problems are always welcome at: dev@hbase.apache.org

Thanks to all who contributed and made this release possible.

Cheers,
The HBase Dev Team

[jira] [Created] (HBASE-25589) release hbase-operator-tools 1.1.0

[Sean Busbey (Jira)]

Sean Busbey created HBASE-25589:
---

 Summary: release hbase-operator-tools 1.1.0
 Key: HBASE-25589
 URL: https://issues.apache.org/jira/browse/HBASE-25589
 Project: HBase
  Issue Type: Task
  Components: community, hbase-operator-tools
Reporter: Sean Busbey
Assignee: Sean Busbey






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] hbase-operator-tools 1.1.0 (RC0)

[Sean Busbey]

With three +1s and no other votes, this release passes as Apache HBase
Operator Tools 1.1.0.

Thanks y'all for voting!

On Sat, Feb 13, 2021 at 12:32 PM Sean Busbey  wrote:
>
> Please vote on this Apache hbase operator tools release candidate,
> hbase-operator-tools-1.1.0RC0
>
> The VOTE will remain open for at least 72 hours.
>
> [ ] +1 Release this package as Apache hbase operator tools 1.1.0
> [ ] -1 Do not release this package because ...
>
> The tag to be voted on is 1.1.0RC0:
>
>   https://github.com/apache/hbase-operator-tools/tree/1.1.0RC0
>
> This tag currently points to git reference
>
>   6e7626e4701d3b50c313ce310c04e570dc138dfa
>
> The release files, including signatures, digests, as well as CHANGES.md
> and RELEASENOTES.md included in this RC can be found at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/hbase-operator-tools-1.1.0RC0/
>
> Maven artifacts are available in a staging repository at:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1436/
> Artifacts were signed with the E65E11D40D80DB7C key which can be found in:
>
>   https://dist.apache.org/repos/dist/release/hbase/KEYS
>
> To learn more about Apache hbase operator tools, please see
>
>   http://hbase.apache.org/
>
> Thanks,
> Your HBase Release Manager

Re: [VOTE] hbase-thirdparty 3.5.0 (RC0)

[Sean Busbey]

with three +1s and no other votes this release passes Apache HBase
Thirdparty 3.5.0.

thanks folks!

On Wed, Feb 17, 2021 at 8:20 AM Peter Somogyi  wrote:
>
> Note: The automatic checksum check failed using hbase-vote.sh with
> whitespace mismatch but the contents match.
>
> On Wed, Feb 17, 2021 at 3:08 PM Peter Somogyi  wrote:
>
> > +1 (binding)
> >
> > * Signature: ok
> > * Checksum : failed
> > * Rat check (1.8.0_242): ok
> >  - mvn clean apache-rat:check
> > * Built from source (1.8.0_242): ok
> >  - mvn clean install  -DskipTests
> > * Unit tests pass (1.8.0_242): ok
> >  - mvn package -P runAllTests  -Dsurefire.rerunFailingTestsCount=3
> >
> > Built hbase master branch with hbase-thirdparty:3.5.0 dependency, executed
> > tests in hbase-http module.
> >
> > On Thu, Feb 11, 2021 at 7:04 AM Sean Busbey  wrote:
> >
> >> Please vote on this Apache hbase thirdparty release candidate,
> >> hbase-thirdparty-3.5.0RC0
> >>
> >> The VOTE will remain open for at least 72 hours.
> >>
> >> [ ] +1 Release this package as Apache hbase thirdparty 3.5.0
> >> [ ] -1 Do not release this package because ...
> >>
> >> The tag to be voted on is 3.5.0RC0, which currently points to git ref:
> >>
> >>   f691e59158ee0889b9659958d37145fe5e3b25d5
> >>
> >>   https://github.com/apache/hbase-thirdparty/tree/3.5.0RC0
> >>
> >> The release files, including signatures, digests, as well as CHANGES.md
> >> and RELEASENOTES.md included in this RC can be found at:
> >>
> >>   https://dist.apache.org/repos/dist/dev/hbase/hbase-thirdparty-3.5.0RC0/
> >>
> >> Maven artifacts are available in a staging repository at:
> >>
> >>   https://repository.apache.org/content/repositories/orgapachehbase-1435/
> >>
> >> Artifacts were signed with my key, E65E11D40D80DB7C, which can be found
> >> in:
> >>
> >>   https://dist.apache.org/repos/dist/release/hbase/KEYS
> >>
> >> To learn more about Apache hbase thirdparty, please see
> >>
> >>   http://hbase.apache.org/
> >>
> >> Thanks,
> >> Your HBase Release Manager
> >>
> >

[jira] [Created] (HBASE-25580) Release scripts should include in the vote email the git hash that the RC tag points to

[Sean Busbey (Jira)]

Sean Busbey created HBASE-25580:
---

 Summary: Release scripts should include in the vote email the git 
hash that the RC tag points to
 Key: HBASE-25580
 URL: https://issues.apache.org/jira/browse/HBASE-25580
 Project: HBase
  Issue Type: Task
  Components: community
Reporter: Sean Busbey
Assignee: Sean Busbey


the git tags for release candidates are not protected, so they could change 
after a RM posts a release candidate. the vote text should include the git hash 
that the RC tag points to at the time of the vote so a change can easily be 
seen.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)