[jira] [Created] (HBASE-21284) Forward port HBASE-21000 to branch-2
Andrew Purtell created HBASE-21284: -- Summary: Forward port HBASE-21000 to branch-2 Key: HBASE-21284 URL: https://issues.apache.org/jira/browse/HBASE-21284 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell See parent for details. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: RMs have to manually update hbase.apache.org/downloads.html now?
I think we got told off for having direct link when trying to send an ANNOUNCE email one time. Not sure which release it was. On Tue, Oct 9, 2018 at 5:33 PM Andrew Purtell wrote: > Thanks. There are a handful of unavoidably manual steps post make_rc.sh. > Updating and rebuilding site is not a problem as long as the workflow steps > are documented. > > > On Tue, Oct 9, 2018 at 3:16 PM Stack wrote: > > > On Tue, Oct 9, 2018 at 2:13 PM Andrew Purtell > wrote: > > > > > It appears RMs have to manually update page source for > > > hbase.apache.org/downloads.html now. Is this correct? > > > > > > Yes. Release is now split between apache.org and mirror sites with > hashes > > and sigs on former and artifacts on the latter rather than as it was in > the > > past with all on the mirror sites. > > > > > > > If so, > > > release instructions at http://hbase.apache.org/book.html#releasing > have > > > not been updated to reflect this. > > > > > > > > Let me have a go at it. > > > > > > > > > No more direct link to https://www.apache.org/dyn/closer.lua/hbase for > > > discovery of all current releases? Can we put it back? > > > > > > > I think we could put it back long as there were a pointer to the download > > page (HEADER.html doesn't render when I browse to > > http://apache.spinellicreations.com/hbase/... whats up w/ that? Do we > need > > an index page? Our HEADER.html could get an update pointing to download > > page too...). > > > > > > S > > > > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > >- A23, Crosstalk > > > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
Re: RMs have to manually update hbase.apache.org/downloads.html now?
Thanks. There are a handful of unavoidably manual steps post make_rc.sh. Updating and rebuilding site is not a problem as long as the workflow steps are documented. On Tue, Oct 9, 2018 at 3:16 PM Stack wrote: > On Tue, Oct 9, 2018 at 2:13 PM Andrew Purtell wrote: > > > It appears RMs have to manually update page source for > > hbase.apache.org/downloads.html now. Is this correct? > > > Yes. Release is now split between apache.org and mirror sites with hashes > and sigs on former and artifacts on the latter rather than as it was in the > past with all on the mirror sites. > > > > If so, > > release instructions at http://hbase.apache.org/book.html#releasing have > > not been updated to reflect this. > > > > > Let me have a go at it. > > > > > No more direct link to https://www.apache.org/dyn/closer.lua/hbase for > > discovery of all current releases? Can we put it back? > > > > I think we could put it back long as there were a pointer to the download > page (HEADER.html doesn't render when I browse to > http://apache.spinellicreations.com/hbase/... whats up w/ that? Do we need > an index page? Our HEADER.html could get an update pointing to download > page too...). > > > S > > > > > > > -- > > Best regards, > > Andrew > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > decrepit hands > >- A23, Crosstalk > > > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: RMs have to manually update hbase.apache.org/downloads.html now?
On Tue, Oct 9, 2018 at 2:13 PM Andrew Purtell wrote: > It appears RMs have to manually update page source for > hbase.apache.org/downloads.html now. Is this correct? Yes. Release is now split between apache.org and mirror sites with hashes and sigs on former and artifacts on the latter rather than as it was in the past with all on the mirror sites. > If so, > release instructions at http://hbase.apache.org/book.html#releasing have > not been updated to reflect this. > > Let me have a go at it. > No more direct link to https://www.apache.org/dyn/closer.lua/hbase for > discovery of all current releases? Can we put it back? > I think we could put it back long as there were a pointer to the download page (HEADER.html doesn't render when I browse to http://apache.spinellicreations.com/hbase/... whats up w/ that? Do we need an index page? Our HEADER.html could get an update pointing to download page too...). S > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands >- A23, Crosstalk >
RMs have to manually update hbase.apache.org/downloads.html now?
It appears RMs have to manually update page source for hbase.apache.org/downloads.html now. Is this correct? If so, release instructions at http://hbase.apache.org/book.html#releasing have not been updated to reflect this. No more direct link to https://www.apache.org/dyn/closer.lua/hbase for discovery of all current releases? Can we put it back? -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
[jira] [Created] (HBASE-21283) Add new shell command 'rit' for listing regions in transition
Andrew Purtell created HBASE-21283:
--
Summary: Add new shell command 'rit' for listing regions in
transition
Key: HBASE-21283
URL: https://issues.apache.org/jira/browse/HBASE-21283
Project: HBase
Issue Type: Improvement
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 3.0.0, 1.5.0, 2.2.0
The 'status' shell command shows regions in transition but sometimes an
operator may want to retrieve a simple list of regions in transition. Here's a
patch that adds a new 'rit' command to the TOOLS group that does just that.
No test, because it seems hard to mock RITs from the ruby test code, but I have
run TestShell and it passes, so the command is verified to meet minimum
requirements, like help text, and manually verified with branch-1 (shell in
branch-2 and up doesn't return until TransitRegionProcedure has completed so by
that time no RIT):
{noformat}
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 1.5.0-SNAPSHOT, r9bb6d2fa8b760f16cd046657240ebd4ad91cb6de, Mon Oct 8
21:05:50 UTC 2018
hbase(main):001:0> help 'rit'
List all regions in transition.
Examples:
hbase> rit
hbase(main):002:0> create ...
0 row(s) in 2.5150 seconds
=> Hbase::Table - IntegrationTestBigLinkedList
hbase(main):003:0> rit
0 row(s) in 0.0340 seconds
hbase(main):004:0> unassign '56f0c38c81ae453d19906ce156a2d6a1'
0 row(s) in 0.0540 seconds
hbase(main):005:0> rit
IntegrationTestBigLinkedList,L\xCC\xCC\xCC\xCC\xCC\xCC\xCB,1539117183224.56f0c38c81ae453d19906ce156a2d6a1.
state=PENDING_CLOSE, ts=Tue Oct 09 20:33:34 UTC 2018 (0s ago), server=null
1 row(s) in 0.0170 seconds
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
HBase quarterly report Jul-Sep 2018
All, HBase submits a report to the ASF board once a quarter, to inform the board about project health. I'm sending the report to the user@ and dev@ mailing lists because you are the project. If you have any questions about the report or the running of the project, you can pose them to me or any other PMC member or committer, or send an email to priv...@hbase.apache.org, which every PMC member subscribes to. Thanks, Misty -- Apache HBase is an open-source, distributed, versioned, non-relational database. Apache HBase gives you low latency random access to billions of rows with millions of columns atop non-specialized hardware. hbase-thirdparty is a set of internal artifacts used by the project to mitigate the impact of our dependency choices on the wider ecosystem. ISSUES FOR THE BOARD’S ATTENTION Board-only information removed from public report. RELEASES - 2.1.0 was released on Tue Jul 17 2018 - 1.4.6 was released on Sun Jul 29 2018 - 2.0.2 was released on Sat Sep 01 2018 - 1.4.7 was released on Sun Sep 02 2018 - 1.2.7 was released on Sat Sep 15 2018 ACTIVITY The second annual HBaseCon Asia was held in Beijing, China, on Aug. 17, 2018. The event was hosted by Alibaba, sponsored by Didi, Huawei, and Xiaomi. Admission was free to attendees, and a live stream allowed worldwide participation. In total, 494 individuals attended locally and the livestream accumulated 14796 views. Thanks to Yu Li and all organizers for coordinating the event. [Event summary]( http://mail-archives.apache.org/mod_mbox/hbase-user/201808.mbox/) [Photos, videos, slides]( http://mail-archives.apache.org/mod_mbox/hbase-user/201808.mbox/) A developer meet-up occurred in conjunction with HBaseCon Asia on August 18, 2018. Around 30 people attended. [Michael Stack took notes]( http://mail-archives.apache.org/mod_mbox/hbase-dev/201808.mbox/browser). Key discussion points included testing, proposals for using Github in our workflow, a discussion about user experienc for new HBase users, and more. Three more developer meet-ups occurred in September in China: - September 1 in HangZhou - September 8 in ShangHai - September 15 in ShenZhen More than 200 people attended locally and hundreds participated via livestreams. [Details and recordings](https://s.apache.org/MdSV) are available (in Chinese). In total, HBase had 5 releases over this quarter, across 4 release lines. Toshihiro Suzuki was added as a committer on Wed Aug 01 2018. Zach York joined the PMC on October 5, 2018 (out of this reporting period). Stay tuned for more committer and PMC member announcements next month, as they just missed the report cut-off date! Thanks to the new committers and PMC members for agreeing to take on more responsibilities in the project. STATS The dev@ mailing list saw a negligible decline in subscribers and nearly identical traffic as the last quarter. The user@ mailing list lost 23 subscribers along with a negligible increase in traffic over the last quarter. 76 committers (+1 from last quarter) 42 PMC members (+0 from last quarter) 413 JIRA tickets created (down from 497 last quarter) 343 JIRA tickets closed/resolved (down from 428 last quarter)
[jira] [Created] (HBASE-21282) Upgrade Jetty dependencies to latest in major-line
Josh Elser created HBASE-21282: -- Summary: Upgrade Jetty dependencies to latest in major-line Key: HBASE-21282 URL: https://issues.apache.org/jira/browse/HBASE-21282 Project: HBase Issue Type: Task Components: dependencies Reporter: Josh Elser Assignee: Josh Elser Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2 Looks like we have dependencies on both jetty 9.2 and 9.3, but we're lagging pretty far behind in both. We can upgrade both of these to the latest (august 2018). I'll also have to take a look at why we're using two separate versions (maybe we didn't want to switch from jetty-jsp to apache-jsp on 9.2->9.3?). Not sure if there's a good reason for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21281) Update bouncycastle dependency.
Josh Elser created HBASE-21281: -- Summary: Update bouncycastle dependency. Key: HBASE-21281 URL: https://issues.apache.org/jira/browse/HBASE-21281 Project: HBase Issue Type: Task Components: test Reporter: Josh Elser Assignee: Josh Elser Fix For: 3.0.0, 2.2.0, 2.0.3, 2.1.2 Looks like we still depend on bcprov-jdk16 for some x509 certificate generation in our tests. Bouncycastle has moved beyond this in 1.47, changing the artifact names. [http://www.bouncycastle.org/wiki/display/JA1/Porting+from+earlier+BC+releases+to+1.47+and+later] There are some API changes too, but it looks like we don't use any of these. It seems like we also have vestiges in the POMs from when we were depending on a specific BC version that came in from Hadoop. We now have a KeyStoreTestUtil class in HBase, which makes me think we can also clean up some dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Does Hbase backup process support encryption while transporting the data from one cluster to other cluster
Hi neo0731, depending on what you mean by encryption, it can mean Hadoop at-rest encryption, Hadoop data transport encryption, Hadoop RPC necryption, HBase data transport encryption, Hbase RPC encryption, SSL encryption (Hadoop WebHDFS, HBase Thrift) distcp supports Hadoop at-rest encryption, and it supports Hadoop data transport encryption/Hadoop RPC encryption. There's a proposal to support webhdfs styled URL in distcp, and that way it will be able to support SSL encrypted traffic. HBase replication supports Hadoop at-rest encryption. We've seen the use case in production. I am aware of some bugs in HBase backup tool that can't read Hadoop encryption zone files. On Tue, Oct 9, 2018 at 5:28 AM neo0731 wrote: > > Question arises when migrating the data from one hbase table to another. > > Input > > To sync the production cluster data with dev cluster. Additionaly, while > copying we need to re-hash the following fields: hashed_email, lexer_id, > foo_imsi, foo_msn, signal_uid, bar_imsi. > > Question is : Does copyTable support hashing of data while copying? Same > for > distcp utility ? Is it possible to supply some example code in scala as > well > > Any help on it would be much appreciated? > > > > -- > Sent from: > http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html > -- A very happy Clouderan
Re: Does Hbase backup process support encryption while transporting the data from one cluster to other cluster
bq. Does copyTable support hashing of data while copying? No. bq. Same for distcp utility ? The above would get better answer posting on hadoop mailing list. Thanks On Tue, Oct 9, 2018 at 5:28 AM neo0731 wrote: > > Question arises when migrating the data from one hbase table to another. > > Input > > To sync the production cluster data with dev cluster. Additionaly, while > copying we need to re-hash the following fields: hashed_email, lexer_id, > foo_imsi, foo_msn, signal_uid, bar_imsi. > > Question is : Does copyTable support hashing of data while copying? Same > for > distcp utility ? Is it possible to supply some example code in scala as > well > > Any help on it would be much appreciated? > > > > -- > Sent from: > http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html >
[jira] [Resolved] (HBASE-20837) Make IDE configuration for import order match that in our checkstyle module
[
https://issues.apache.org/jira/browse/HBASE-20837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak Lon (Stephen) Wu resolved HBASE-20837.
--
Resolution: Won't Fix
> Make IDE configuration for import order match that in our checkstyle module
> ---
>
> Key: HBASE-20837
> URL: https://issues.apache.org/jira/browse/HBASE-20837
> Project: HBase
> Issue Type: Improvement
> Components: community
>Affects Versions: 3.0.0, 2.0.1, 1.4.5
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-20837.branch-1.001.patch,
> HBASE-20837.branch-1.002.patch, HBASE-20837.branch-1.003.patch,
> HBASE-20837.branch-2.001.patch, HBASE-20837.branch-2.002.patch,
> HBASE-20837.branch-2.003.patch, HBASE-20837.master.001.patch,
> HBASE-20837.master.002.patch, HBASE-20837.master.003.patch, IDEA import
> layout.png, hbase-intellij-formatter.xml
>
>
> While working on HBASE-20557 contribution, we figured out that the checkstyle
> build target (ImportOrder's `groups`
> [http://checkstyle.sourceforge.net/config_imports.html] ) was different from
> the development supported IDE (e.g. IntelliJ and Eclipse) formatter, we would
> provide a fix here to sync between
> [dev-support/hbase_eclipse_formatter.xml|https://github.com/apache/hbase/blob/master/dev-support/hbase_eclipse_formatter.xml]
> and
> [hbase/checkstyle.xml|https://github.com/apache/hbase/blob/master/hbase-checkstyle/src/main/resources/hbase/checkstyle.xml]
> This might need to backport the changes of master to branch-1 and branch-2 as
> well.
> Before this change, this is what checkstyle is expecting for import order
>
> {code:java}
> import com.google.common.annotations.VisibleForTesting;
> import java.io.IOException;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.Map;
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.hbase.classification.InterfaceAudience;
> import org.apache.hadoop.hbase.conf.ConfigurationObserver;{code}
>
> And the proposed import order with the respect to HBASE-19262 and HBASE-19552
> should be
>
> !IDEA import layout.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21280) Add anchors for each heading in UI
stack created HBASE-21280: - Summary: Add anchors for each heading in UI Key: HBASE-21280 URL: https://issues.apache.org/jira/browse/HBASE-21280 Project: HBase Issue Type: Bug Components: UI, Usability Reporter: stack Assignee: stack On larger clusters, its annoying having to scroll down on each refresh. Anchors would help pin page to a section in UI (until our UI gets redone...) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Does Hbase backup process support encryption while transporting the data from one cluster to other cluster
Question arises when migrating the data from one hbase table to another. Input To sync the production cluster data with dev cluster. Additionaly, while copying we need to re-hash the following fields: hashed_email, lexer_id, foo_imsi, foo_msn, signal_uid, bar_imsi. Question is : Does copyTable support hashing of data while copying? Same for distcp utility ? Is it possible to supply some example code in scala as well Any help on it would be much appreciated? -- Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html
[jira] [Resolved] (HBASE-21235) Rename the closed procedure wal files so that we do not need to call recoverLease when restarting
[ https://issues.apache.org/jira/browse/HBASE-21235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-21235. --- Resolution: Not A Problem If we can limit the total number of proc wal files then this is useless. Close as not a problem. > Rename the closed procedure wal files so that we do not need to call > recoverLease when restarting > - > > Key: HBASE-21235 > URL: https://issues.apache.org/jira/browse/HBASE-21235 > Project: HBase > Issue Type: Sub-task > Components: Performance, proc-v2 >Reporter: Duo Zhang >Priority: Major > > If there are lots of procedure wal files the recover lease will be a time > consuming operation. Renaming is a possible way to confirm that some files > are already closed when restarting so we do not need to call recoverLease on > them any more. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
