[jira] [Created] (HBASE-25210) RegionInfo.isOffline is now a duplication with RegionInfo.isSplit

[Duo Zhang (Jira)]

Duo Zhang created HBASE-25210:
-

 Summary: RegionInfo.isOffline is now a duplication with 
RegionInfo.isSplit
 Key: HBASE-25210
 URL: https://issues.apache.org/jira/browse/HBASE-25210
 Project: HBase
  Issue Type: Improvement
Reporter: Duo Zhang


The only place, where we set it to true is in splitRegion, and at the same time 
we will set split to true.

So in general, I suggest that we deprecated isOffline and isSplitParent in 
RegionInfo, only leave the isSplit method. And in RegionInfoBuilder, we 
deprecated setOffline and only leave the setSplit method.

This could make our code base cleaner.

And for serialization compatibility, we'd better still keep the split and 
offline fields in the actual RegionInfo datastructure for a while.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] the IA annotation on hbase client exceptions

[Yulin Niu]

So, we introduce a new Annotation IA.LimitedPrivate(Exception) to decorate
the exceptions, which are free to catched and propagated by users, but
should not be created by users themselves.

张铎(Duo Zhang)  于2020年10月18日周日 下午10:08写道：

> I think IA.Public maybe over killed here for some exceptions.
> For example, the DoNotRetryIOException, we just want users to catch this
> exception, but usually we do not expect users to create this exeption by
> their own?
> But IA.Public means we can not change the public methods of the class, so
> we can not even remove constructors of the exception?
>
> Maybe special IA.LimitedPrivate(Exception)? Which means you are free to
> catch and propagate the exception, but you should not create it by
> yourselves? Or use the special methods of the exception?
>
> Sean Busbey  于2020年10月18日周日 下午9:46写道：
>
> > My guess would be IA.Public is needed if the client can take action based
> > on the specific exception. So like the do not retry exception should be
> > public so folks know those IOExceptions that it's not worth baking off
> and
> > retrying.
> >
> > What's the specific list of public vs not? Why do you want the non public
> > ones to be public? What limitations are we putting on clients by keeping
> > them private?
> >
> > On Sun, Oct 18, 2020, 07:45 Yulin Niu  wrote:
> >
> > > Hi, guys:
> > >
> > > There are IA.Public and also IA.Private under the hbase-client
> > exceptions
> > > package, whether the IA.Private exceptions should be IA.Public?
> > > e.g.MasterRegistryFetchException, OutOfOrderScannerNextException
> > > ,RegionMovedException,RegionOpeningException
> > > And any others thing about the IA annotation on exceptions?
> > > FYI 2538 
> > >
> > > Thanks
> > >
> >
>

[jira] [Resolved] (HBASE-25209) Add CLI support to remove a particular configuration from ReplicationPeerConfig.

[Ankit Jain (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Jain resolved HBASE-25209.

Resolution: Not A Problem

> Add CLI support to remove a particular configuration from 
> ReplicationPeerConfig.
> 
>
> Key: HBASE-25209
> URL: https://issues.apache.org/jira/browse/HBASE-25209
> Project: HBase
>  Issue Type: Improvement
>  Components: Replication, shell
> Environment: {code:java}
> // code placeholder
> {code}
>Reporter: Ankit Jain
>Assignee: Ankit Jain
>Priority: Minor
>
> Currently, there is no easy way to remove a configuration from 
> ReplicationPeerConfig after it is being set.
> This can come in handy at multiple instances when we want to remove a 
> particular configuration when it's not needed anymore or it is causing 
> unexpected behavior. For eg in case there is a need to remove a walFilter 
> (hbase.replication.source.custom.walentryfilters) configuration.
> As part of this Jira we want to add that support.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25209) Add CLI support to remove a particular configuration from ReplicationPeerConfig.

[Ankit Jain (Jira)]

Ankit Jain created HBASE-25209:
--

 Summary: Add CLI support to remove a particular configuration from 
ReplicationPeerConfig.
 Key: HBASE-25209
 URL: https://issues.apache.org/jira/browse/HBASE-25209
 Project: HBase
  Issue Type: Improvement
  Components: Replication, shell
 Environment: {code:java}
// code placeholder
{code}
Reporter: Ankit Jain
Assignee: Ankit Jain


Currently, there is no easy way to remove a configuration from 
ReplicationPeerConfig after it is being set.

This can come in handy at multiple instances when we want to remove a 
particular configuration when it's not needed anymore or it is causing 
unexpected behavior. For eg in case there is a need to remove a walFilter 
(hbase.replication.source.custom.walentryfilters) configuration.

As part of this Jira we want to add that support.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25208) Running an hbase-shell command in non-interactive mode should not complain about missing files when it fails to talk to the cluster.

[Sean Busbey (Jira)]

Sean Busbey created HBASE-25208:
---

 Summary: Running an hbase-shell command in non-interactive mode 
should not complain about missing files when it fails to talk to the cluster.
 Key: HBASE-25208
 URL: https://issues.apache.org/jira/browse/HBASE-25208
 Project: HBase
  Issue Type: Bug
  Components: shell
Reporter: Sean Busbey


if you script out some shell commands, then run them via the shell in 
non-interactive mode we sometimes give an erroneous "I couldn't find that file" 
response when the master(s) are in a bad state instead of revealing the 
underlying cluster issue.

e.g.
{code}
  tmpfile=$(mktemp)
  echo "balance_switch false" > $tmpfile
  echo "exit" >> $tmpfile
  $HBASE_BIN --config $CONF_DIR shell -n $tmpfile
  RET=$?
  rm $tmpfile
  exit $RET
{code}

as an example, with a master stuck trying to initialize as active the failure 
might look like

{code}
++ mktemp
+ tmpfile=/tmp/tmp.dMHTj0nVXx
+ echo balance_switch true
+ echo exit
+ bin/hbase --config /etc/hbase/conf shell -n /tmp/tmp.dMHTj0nVXx
LoadError: no such file to load -- /tmp/tmp.dMHTj0nVXx
load at org/jruby/RubyKernel.java:974
   at /some/path/to/lib/hbase/bin/hirb.rb:186
+ RET=1
+ rm /tmp/tmp.dMHTj0nVXx
+ exit 1
{code}

which is very confusing. the shell should report something related to an issue 
talking to the cluster rather than a problem with the passed script.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25207) Revisit the implementation and usage of RegionStates.include

[Duo Zhang (Jira)]

Duo Zhang created HBASE-25207:
-

 Summary: Revisit the implementation and usage of 
RegionStates.include
 Key: HBASE-25207
 URL: https://issues.apache.org/jira/browse/HBASE-25207
 Project: HBase
  Issue Type: Bug
Reporter: Duo Zhang


After several round of refactoring and fixing, the method has been used in lots 
of places and the implementation looks really confusing.

As in the first if condition for testing RegionStateNode and RegionInfo state, 
we will always return false when split is true, which means we will always 
filter out split parent, as a split parent, is split = true and also offline = 
true.

I think the reason why there is no problem is that, only in 
EnableTableProcedure we call this method with offline = true, and 
EnableTableProcedure does not need to deal with split parent...

And now since we found a problem in HBASE-25206, where we need to get split 
parent when deleting a table, I think it is time to revisit this method and 
make logic less confusing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-25206) Data loss can happen if a cloned table loses original split region(delete table)

[Toshihiro Suzuki (Jira)]

Toshihiro Suzuki created HBASE-25206:


 Summary: Data loss can happen if a cloned table loses original 
split region(delete table)
 Key: HBASE-25206
 URL: https://issues.apache.org/jira/browse/HBASE-25206
 Project: HBase
  Issue Type: Bug
Reporter: Toshihiro Suzuki
Assignee: Toshihiro Suzuki


Steps to reproduce are as follows:

1. Create a table and put some data into the table:
{code:java}
create 'test1','cf'
put 'test1','r1','cf','v1'
put 'test1','r2','cf','v2'
put 'test1','r3','cf','v3'
put 'test1','r4','cf','v4'
put 'test1','r5','cf','v5'
{code}
2. Take a snapshot for the table:
{code:java}
snapshot 'test1','snap_test'
{code}
3. Clone the snapshot to another table
{code:java}
clone_snapshot 'snap_test','test2'
{code}
4. Delete the snapshot
{code:java}
delete_snapshot 'snap_test'
{code}
5. Split the original table
{code:java}
split 'test1','r3'
{code}
6. Drop the original table
{code:java}
disable 'test1'
drop 'test1'
{code}
After that, we see the error like the following in RS log when opening the 
regions of the cloned table:
{code:java}
2020-10-20 22:15:47,554 WARN  [RS_OPEN_REGION-regionserver/10.0.1.8:0-0] 
regionserver.HRegion(965): Failed initialize of region= 
testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_-1603199739880,,1603199732706.92f431fab12aaded92a23513901daa5a.,
 starting to roll back memstore
java.io.IOException: java.io.IOException: java.io.FileNotFoundException: 
HFileLink 
locations=[hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/.tmp/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/mobdir/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/archive/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402]
at 
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1179)
at 
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1121)
at 
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1011)
at 
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:962)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7999)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7955)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7930)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7888)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7839)
at 
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:132)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: java.io.FileNotFoundException: HFileLink 
locations=[hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/.tmp/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 
hdfs://localhost:62716/user/tsuzuki/test-data/c00e6c6b-1c3b-5e40-4227-831ae42cf2f4/mobdir/data/default/testCloneSnapshotBeforeSplittingRegionAndDroppingTable_0__regionReplication_1_1603199732705/f4658c2b6fb129d95f62e63d3742177d/cf/719b64120a0f4394ae7af8926bc56402,
 

[jira] [Resolved] (HBASE-25204) Nightly job failed as the name of jdk and maven changed

[Guanghao Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guanghao Zhang resolved HBASE-25204.

Fix Version/s: 2.2.7
   1.4.14
   2.4.0
   1.7.0
   2.3.3
   3.0.0-alpha-1
   Resolution: Fixed

Pushed to all active branchs. Thanks [~zhangduo] for reviewing.

> Nightly job failed as  the name of jdk and maven changed
> 
>
> Key: HBASE-25204
> URL: https://issues.apache.org/jira/browse/HBASE-25204
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> See 
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console]
> [https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console]
>  
> org.codehaus.groovy.control.MultipleCompilationErrorsException: startup 
> failed: WorkflowScript: 508: Tool type "maven" does not have an install of 
> "Maven (latest)" configured - did you mean "maven_latest"? @ line 508, column 
> 19. maven 'Maven (latest)' ^ WorkflowScript: 510: Tool type "jdk" does not 
> have an install of "JDK 1.8 (latest)" configured - did you mean 
> "jdk_1.8_latest"? @ line 510, column 17. jdk "JDK 1.8 (latest)"
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25203) Change the reference url to flaky list in our jenkins jobs

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-25203.
---
Hadoop Flags: Reviewed
  Resolution: Fixed

Pushed to all active branches.

Thanks [~zghao] for reviewing.

> Change the reference url to flaky list in our jenkins jobs
> --
>
> Key: HBASE-25203
> URL: https://issues.apache.org/jira/browse/HBASE-25203
> Project: HBase
>  Issue Type: Sub-task
>  Components: flakies, jenkins
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
> Nightly
> Flaky tests
> PreCommit
> personality



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: branch-2.2/branch-2.3 nightly jon failed to run

[Guanghao Zhang]

Opened HBASE-25204 to fix this.

张铎(Duo Zhang)  于2020年10月20日周二 下午12:40写道：

> Master is failing too, the same problem.
>
>
> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/98/console
>
> I think it is what Nick said, the name of jdk and maven seems to be
> changed. Mind opening an issue to fix it Guanghao?
>
> Thanks.
>
>
>
> Sean Busbey  于2020年10月20日周二 上午11:42写道：
>
> > that sounds probable given the symptoms.
> >
> > On Mon, Oct 19, 2020 at 9:16 PM Nick Dimiduk 
> wrote:
> > >
> > > The Jenkins server was restarted within the last 24 hours, presumably
> for
> > > maintenance. Maybe they changed the name of this binary mapping without
> > > telling us?
> > >
> > > On Mon, Oct 19, 2020 at 17:42 Guanghao Zhang 
> wrote:
> > >
> > > > See
> > > >
> > > >
> >
> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/85/console
> > > >
> > > >
> >
> https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/103/console
> > > >
> > > > org.codehaus.groovy.control.MultipleCompilationErrorsException:
> startup
> > > > failed:
> > > > WorkflowScript: 508: Tool type "maven" does not have an install of
> > > > "Maven (latest)" configured - did you mean "maven_latest"? @ line
> 508,
> > > > column 19.
> > > >maven 'Maven (latest)'
> > > >  ^
> > > >
> > > > WorkflowScript: 510: Tool type "jdk" does not have an install of "JDK
> > > > 1.8 (latest)" configured - did you mean "jdk_1.8_latest"? @ line 510,
> > > > column 17.
> > > >jdk "JDK 1.8 (latest)"
> > > >
> > > >
> > > > But the Jenkinsfile did not change recently. Any ideas?
> > > >
> >
> >
> >
> > --
> > Sean
> >
>