[jira] [Resolved] (HBASE-25303) Nightly build setup failure with Ruby version
[
https://issues.apache.org/jira/browse/HBASE-25303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan resolved HBASE-25303.
---
Resolution: Resolved
> Nightly build setup failure with Ruby version
> -
>
> Key: HBASE-25303
> URL: https://issues.apache.org/jira/browse/HBASE-25303
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.2.7
>Reporter: Peter Somogyi
>Priority: Critical
> Fix For: 1.7.0
>
>
> Nightly builds are failing on branch-2.2 and branch-1 in Docker setup phase.
> {noformat}
> Step 28/33 : RUN gem install rubocop:'<= 0.81'
> ---> Running in ca2c6ed8ee8a
> Building native extensions. This could take a while...
> ERROR: Error installing rubocop:
> parallel requires Ruby version >= 2.5.
> Successfully installed jaro_winkler-1.5.4
> The command '/bin/sh -c gem install rubocop:'<= 0.81'' returned a non-zero
> code: 1
> ERROR: Docker failed to build yetus/hbase:e5c0a9422c.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25303) Nightly build setup failure with Ruby version
[
https://issues.apache.org/jira/browse/HBASE-25303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25303:
--
Fix Version/s: 1.7.0
> Nightly build setup failure with Ruby version
> -
>
> Key: HBASE-25303
> URL: https://issues.apache.org/jira/browse/HBASE-25303
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.7.0, 2.2.7
>Reporter: Peter Somogyi
>Priority: Critical
> Fix For: 1.7.0
>
>
> Nightly builds are failing on branch-2.2 and branch-1 in Docker setup phase.
> {noformat}
> Step 28/33 : RUN gem install rubocop:'<= 0.81'
> ---> Running in ca2c6ed8ee8a
> Building native extensions. This could take a while...
> ERROR: Error installing rubocop:
> parallel requires Ruby version >= 2.5.
> Successfully installed jaro_winkler-1.5.4
> The command '/bin/sh -c gem install rubocop:'<= 0.81'' returned a non-zero
> code: 1
> ERROR: Docker failed to build yetus/hbase:e5c0a9422c.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-11290) Unlock RegionStates
[ https://issues.apache.org/jira/browse/HBASE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-11290: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Unlock RegionStates > --- > > Key: HBASE-11290 > URL: https://issues.apache.org/jira/browse/HBASE-11290 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.2.0, 1.3.0 >Reporter: Francis Christopher Liu >Assignee: Francis Christopher Liu >Priority: Major > Fix For: 1.8.0 > > Attachments: HBASE-11290-0.98.patch, HBASE-11290-0.98_v2.patch, > HBASE-11290.01.branch-1.patch, HBASE-11290.02.patch, HBASE-11290.03.patch, > HBASE-11290.draft.patch, HBASE-11290_trunk.patch > > > Even though RegionStates is a highly accessed data structure in HMaster. Most > of it's methods are synchronized. Which limits concurrency. Even simply > making some of the getters non-synchronized by using concurrent data > structures has helped with region assignments. We can go as simple as this > approach or create locks per region or a bucket lock per region bucket. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-11685) Incr/decr on the reference count of HConnectionImplementation need be atomic
[
https://issues.apache.org/jira/browse/HBASE-11685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-11685:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Incr/decr on the reference count of HConnectionImplementation need be atomic
> -
>
> Key: HBASE-11685
> URL: https://issues.apache.org/jira/browse/HBASE-11685
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: Shaohui Liu
>Assignee: Shaohui Liu
>Priority: Minor
> Fix For: 1.8.0
>
> Attachments: HBASE-11685-trunk-v1.diff, HBASE-11685-trunk-v2.diff,
> HBASE-11685-trunk-v3.diff, HBASE-11685-trunk-v4.diff,
> HBASE-11685-trunk-v5.diff, HBASE-11685-trunk-v6.diff
>
>
> Currently, the incr/decr operation on the ref count of
> HConnectionImplementation are not atomic. This may cause that the ref count
> always be larger than 0 and the connection never be closed.
> {code}
> /**
> * Increment this client's reference count.
> */
> void incCount() {
> ++refCount;
> }
> /**
> * Decrement this client's reference count.
> */
> void decCount() {
> if (refCount > 0) {
> --refCount;
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-12943) Set sun.net.inetaddr.ttl in HBase
[ https://issues.apache.org/jira/browse/HBASE-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-12943: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Set sun.net.inetaddr.ttl in HBase > - > > Key: HBASE-12943 > URL: https://issues.apache.org/jira/browse/HBASE-12943 > Project: HBase > Issue Type: Bug >Reporter: Shaohui Liu >Assignee: Shaohui Liu >Priority: Major > Fix For: 3.0.0-alpha-1, 1.8.0 > > Attachments: 12943-1-master.txt > > > The default value of config: sun.net.inetaddr.ttl is -1 and the java > processes will cache the mapping of hostname to ip address forever, See: > http://docs.oracle.com/javase/7/docs/technotes/guides/net/properties.html > But things go wrong when a regionserver with same hostname and different ip > address rejoins the hbase cluster. The HMaster will get wrong ip address of > the regionserver from this cache and every region assignment to this > regionserver will be blocked for a time because the HMaster can't communicate > with the regionserver. > A tradeoff is to set the sun.net.inetaddr.ttl to 10m or 1h and make the wrong > cache expired. > Suggestions are welcomed. Thanks~ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-13160) SplitLogWorker does not pick up the task immediately
[
https://issues.apache.org/jira/browse/HBASE-13160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-13160:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> SplitLogWorker does not pick up the task immediately
>
>
> Key: HBASE-13160
> URL: https://issues.apache.org/jira/browse/HBASE-13160
> Project: HBase
> Issue Type: Improvement
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Major
> Fix For: 1.8.0
>
> Attachments: hbase-13160_v1.patch
>
>
> We were reading some code with Jeffrey, and we realized that the
> SplitLogWorker's internal task loop is weird. It does {{ls}} every second and
> sleeps, but have another mechanism to learn about new tasks, but does not
> make affective use of the zk notification.
> I have a simple patch which might improve this area.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-14927) Backport HBASE-13014 (Java Tool For Region Moving) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-14927: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Backport HBASE-13014 (Java Tool For Region Moving) to branch-1 > -- > > Key: HBASE-14927 > URL: https://issues.apache.org/jira/browse/HBASE-14927 > Project: HBase > Issue Type: Improvement >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-14964) Backport HBASE-14901 (duplicated code to create/manage encryption keys) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-14964: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Backport HBASE-14901 (duplicated code to create/manage encryption keys) to > branch-1 > --- > > Key: HBASE-14964 > URL: https://issues.apache.org/jira/browse/HBASE-14964 > Project: HBase > Issue Type: Improvement > Components: encryption >Reporter: Nate Edel >Priority: Minor > Fix For: 1.8.0 > > Attachments: HBASE-14964-branch-1.1.patch, > HBASE-14964.1.branch-1.patch, HBASE-14964.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. > A lot of the Mob stuff hasn't been backported, so this is a very small patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-16071) The VisibilityLabelFilter and AccessControlFilter should not count the "delete cell"
[
https://issues.apache.org/jira/browse/HBASE-16071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-16071:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> The VisibilityLabelFilter and AccessControlFilter should not count the
> "delete cell"
>
>
> Key: HBASE-16071
> URL: https://issues.apache.org/jira/browse/HBASE-16071
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 1.8.0
>
> Attachments: HBASE-16071-v1.patch, HBASE-16071-v2.patch,
> HBASE-16071-v3.patch
>
>
> The VisibilityLabelFilter will see and count the "delete cell" if the
> scan.isRaw() returns true, so the (put) cell will be skipped if it has lower
> version than "delete cell"
> The critical code is shown below:
> {code:title=VisibilityLabelFilter.java|borderStyle=solid}
> public ReturnCode filterKeyValue(Cell cell) throws IOException {
> if (curFamily.getBytes() == null
> || !(CellUtil.matchingFamily(cell, curFamily.getBytes(),
> curFamily.getOffset(),
> curFamily.getLength( {
> curFamily.set(cell.getFamilyArray(), cell.getFamilyOffset(),
> cell.getFamilyLength());
> // For this family, all the columns can have max of
> curFamilyMaxVersions versions. No need to
> // consider the older versions for visibility label check.
> // Ideally this should have been done at a lower layer by HBase (?)
> curFamilyMaxVersions = cfVsMaxVersions.get(curFamily);
> // Family is changed. Just unset curQualifier.
> curQualifier.unset();
> }
> if (curQualifier.getBytes() == null
> || !(CellUtil.matchingQualifier(cell, curQualifier.getBytes(),
> curQualifier.getOffset(),
> curQualifier.getLength( {
> curQualifier.set(cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength());
> curQualMetVersions = 0;
> }
> curQualMetVersions++;
> if (curQualMetVersions > curFamilyMaxVersions) {
> return ReturnCode.SKIP;
> }
> return this.expEvaluator.evaluate(cell) ? ReturnCode.INCLUDE :
> ReturnCode.SKIP;
> }
> {code}
> [VisibilityLabelFilter.java|https://github.com/apache/hbase/blob/d7a4499dfc8b3936a0eca867589fc2b23b597866/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityLabelFilter.java]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-18162) Backport HBASE-16196 (Update jruby to a newer version) branch-1
[ https://issues.apache.org/jira/browse/HBASE-18162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18162: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Backport HBASE-16196 (Update jruby to a newer version) branch-1 > --- > > Key: HBASE-18162 > URL: https://issues.apache.org/jira/browse/HBASE-18162 > Project: HBase > Issue Type: Task > Components: dependencies, shell >Affects Versions: 1.4.0 >Reporter: Sean Busbey >Priority: Major > Fix For: 1.8.0 > > > Work on HBASE-16196 ran into shell failures while attempting a branch-1 > backport. > Start from either the attempted backport or the master branch version. Work > out what's different in branch-1 that causes TestShell to fail. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-18415) The local timeout may cause Admin to submit duplicate request
[ https://issues.apache.org/jira/browse/HBASE-18415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-18415: -- Fix Version/s: (was: 1.7.0) 1.8.0 > The local timeout may cause Admin to submit duplicate request > - > > Key: HBASE-18415 > URL: https://issues.apache.org/jira/browse/HBASE-18415 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 3.0.0-alpha-1, 1.8.0 > > Attachments: HBASE-18415.branch-1.ut.patch, > HBASE-18415.branch-1.v0.patch, HBASE-18415.branch-1.v1.patch, > HBASE-18415.branch-1.v2.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v3.patch, HBASE-18415.branch-1.v3.patch, > HBASE-18415.branch-1.v4.patch, HBASE-18415.branch-1.v4.patch, > HBASE-18415.branch-1.v4.patch > > > After a timeout occurs on first request, client will retry the request with > distinct group/nonce. The second request may bring the TableXXXException back > if the first request have changed the table state. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20151) Bug with SingleColumnValueFilter and FamilyFilter
[
https://issues.apache.org/jira/browse/HBASE-20151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20151:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Bug with SingleColumnValueFilter and FamilyFilter
> -
>
> Key: HBASE-20151
> URL: https://issues.apache.org/jira/browse/HBASE-20151
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.1.0, 2.0.1, 1.4.5
> Environment: MacOS 10.13.3
> HBase 1.3.1
>Reporter: Steven Sadowski
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.8.0
>
> Attachments: HBASE-20151.master.001.patch,
> HBASE-20151.master.002.patch, HBASE-20151.master.003.patch,
> HBASE-20151.master.004.patch, HBASE-20151.master.004.patch,
> HBASE-20151.master.005.patch, HBASE-20151.master.006.patch,
> filter-list-type.v1.txt
>
>
> When running the following queries, the result is sometimes return correctly
> and other times incorrectly based on the qualifier queried.
> Setup:
> {code:java}
> create 'test', 'a', 'b'
> test = get_table 'test'
> test.put '1', 'a:1', nil
> test.put '1', 'a:10', nil
> test.put '1', 'b:2', nil
> {code}
>
> This query works fine when the SCVF's qualifier has length 1 (i.e. '1') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','1',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0060 seconds
> {code}
>
> The query should return the same result when passed a qualifier of length 2
> (i.e. '10') :
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) AND
> FamilyFilter(=,'binary:b') )"})
> ROW COLUMN+CELL
> 0 row(s) in 0.0110 seconds
> {code}
> However, in this case, it does not return any row (expected result would be
> to return the same result as the first query).
>
> Removing the family filter while the qualifier is '10' yields expected
> results:
> {code:java}
> test.scan({ FILTER => "(
> SingleColumnValueFilter('a','10',=,'binary:',true,true) )"})
> ROW COLUMN+CELL
> 1column=a:1,
> timestamp=1520455887954, value=
> 1column=a:10,
> timestamp=1520455888024, value=
> 1column=b:2,
> timestamp=1520455888059, value=
> 1 row(s) in 0.0140 seconds
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-20172) During coprocessor load, switch classloader only if it's a custom CP.
[ https://issues.apache.org/jira/browse/HBASE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20172: -- Fix Version/s: (was: 1.7.0) 1.8.0 > During coprocessor load, switch classloader only if it's a custom CP. > - > > Key: HBASE-20172 > URL: https://issues.apache.org/jira/browse/HBASE-20172 > Project: HBase > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Fix For: 1.8.0 > > Attachments: HBASE-20172-branch-1.patch, HBASE-20172.patch > > > Current Impact:- > Metric registries will not be able to load its implementation through service > loader and etc. > We are not observing this with Java8 because ServiceLoader uses system class > loader if provided class loader is null but it gets exposed with Java7 > easily(TEPHRA-285) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20643) Getting HDFSBlockDist in Master by querying RegionServers
[ https://issues.apache.org/jira/browse/HBASE-20643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20643: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Getting HDFSBlockDist in Master by querying RegionServers > - > > Key: HBASE-20643 > URL: https://issues.apache.org/jira/browse/HBASE-20643 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Labels: balancer > Fix For: 3.0.0-alpha-1, 1.8.0 > > > Region locality information is needed by the balancer to generate region > plans. Computing HDFSBlockDistribution is expensive on larger clusters and > adds load to the NameNode. This also needs to be recomputed on a master > restart. The proposal is to get the HDFSBlockDistribution from the > RegionServers instead of computing it in Master. RS already has this > information and we could just reuse it by querying it. RS already passes > dataLocality info via RegionLoad today. > Proposed Implementation: This is a high-level overview. > # A RegionServer API has to be added which will return HDFSBlockDistribution > for all the regions it hosts. RS already has this info. Since ClusterStatus > has already become bulky and we don’t need updated locality so fast, it’s > better to have another API rather than add this to RegionLoad and pass it > along with RSReport. > # Master will have a Chore to query all RegionServers and will cache the > HDFSBlockDistribution for those regions. This is easy and quick. Admins can > tune the frequency based on size of the cluster. On a ~90 nodes cluster with > 500k regions and a prototype implementation and no load, it took about 5 > seconds to get all HDFSBlockDistribution from RS. > # The cache will be an extension of RegionLocationFinder (subclass), if > needed to keep the implementation simple. Probably will get clear with > implementation. > # Balancer will use the new cache to get all HDFSBlockDistribution. If there > is a new region and Chore didn’t get the block distribution from RS during > its previous run, then it will be computed by RegionLocationFinder the same > way it has been done now. If the Chore runs more frequently like every hour, > then this recomputation will be drastically reduced. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20926) IntegrationTestRSGroup is broken
[ https://issues.apache.org/jira/browse/HBASE-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-20926: -- Fix Version/s: (was: 1.7.0) 1.8.0 > IntegrationTestRSGroup is broken > - > > Key: HBASE-20926 > URL: https://issues.apache.org/jira/browse/HBASE-20926 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 2.1.0 >Reporter: Sergey Soldatov >Assignee: Sergey Soldatov >Priority: Major > Fix For: 3.0.0-alpha-1, 1.8.0 > > > There are several problems: > 1. It doesn't work in minicluster mode because in afterMethod() it using > IntegrationTestingUtility.restoreCluster() which just shutdown minicluster in > the not distributed mode > 2. It uses tests from TestRSGroups which was supposed to be common for both > unit and integration tests, but for last two years, there were a number of > tests added that are using internal API, not compatible with the distributed > mode. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-21904) TestSimpleRpcScheduler is still flaky
[ https://issues.apache.org/jira/browse/HBASE-21904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21904: -- Fix Version/s: (was: 1.7.0) 1.8.0 > TestSimpleRpcScheduler is still flaky > - > > Key: HBASE-21904 > URL: https://issues.apache.org/jira/browse/HBASE-21904 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 1.5.0 >Reporter: Andrew Kyle Purtell >Priority: Major > Labels: branch-1 > Fix For: 1.8.0 > > Attachments: > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler-output.txt > > > Flaky wait condition, unclear if it's the wait condition or the underlying > functionality that is the problem. > [ERROR] > testSoftAndHardQueueLimits(org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler) > Time elapsed: 0.228 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.hadoop.hbase.ipc.TestSimpleRpcScheduler.testSoftAndHardQueueLimits(TestSimpleRpcScheduler.java:380) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20993:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 1.2.6, 1.3.2, 1.2.7, 1.4.7
>Reporter: Reid Chan
>Priority: Critical
> Fix For: 1.8.0
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.010.patch, HBASE-20993.branch-1.011.patch,
> HBASE-20993.branch-1.012.patch, HBASE-20993.branch-1.013.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
>
[jira] [Updated] (HBASE-22356) API to get hdfs block distribution from regionservers
[ https://issues.apache.org/jira/browse/HBASE-22356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22356: -- Fix Version/s: (was: 1.7.0) 1.8.0 > API to get hdfs block distribution from regionservers > - > > Key: HBASE-22356 > URL: https://issues.apache.org/jira/browse/HBASE-22356 > Project: HBase > Issue Type: Sub-task > Components: API, Balancer, regionserver >Reporter: Thiruvel Thirumoolan >Assignee: Thiruvel Thirumoolan >Priority: Major > Labels: balancer > Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0 > > Attachments: HBASE-22356.master.001.patch, > HBASE-22356.master.002.patch, HBASE-22356.master.003.patch > > > A RegionServer API has to be added which will return HDFSBlockDistribution > for all the regions it hosts. RS already has this info cached and updated > when flush/compaction happens. Master can query and get instead of hitting > the namenode and caching. The larger the cluster becomes, the more costly it > becomes to get this information and more stale the cached information becomes. > This jira is only to add the API to regionserver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22630) Restore TestReplicationDroppedTables coverage to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22630: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Restore TestReplicationDroppedTables coverage to branch-1 > - > > Key: HBASE-22630 > URL: https://issues.apache.org/jira/browse/HBASE-22630 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Priority: Major > Fix For: 1.8.0 > > > TestReplicationDroppedTables was dropped from branch-1. Restore the test > coverage with a test that is not flaky. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22839) Make sure the batches within one region are shipped to the sink clusters in order (branch-1)
[ https://issues.apache.org/jira/browse/HBASE-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-22839: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Make sure the batches within one region are shipped to the sink clusters in > order (branch-1) > > > Key: HBASE-22839 > URL: https://issues.apache.org/jira/browse/HBASE-22839 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 1.3.4, 1.3.5 >Reporter: Bin Shi >Assignee: Bin Shi >Priority: Major > Fix For: 1.8.0 > > > Problem Statement: > In the cross-cluster replication validation, we found some cells in source > and sink cluster can have the same row key, the same timestamp but different > values. The happens when mutations with the same row key are submitted in > batch without specifying the timestamp, and the same timestamp in the unit of > millisecond is assigned at the time when they are committed to the WAL. > When this happens, if the major compaction hasn’t happened yet and you scan > the table, you can find some cells have the same row key, the same timestamps > but different values, like the first three rows in the following table. > |Row Key 1|CF0::Column 1|Timestatmp 1|Value 1| > |Row Key 1|CF0::Column 1|Timestatmp 1|Value 2| > |Row Key 1|CF0::Column 1|Timestatmp 1|Value 3| > |Row Key 2|CF0::Column 1|Timestatmp 2|Value 4| > |Row Key 3|CF0::Column 1|Timestatmp 4|Value 5| > The ordering of the first three rows is indeterminate in the presence of the > cross-replication, so after compaction, in the master cluster you will see > “Row Key 1, CF0::Column1, Timestamp1” having the value 3, but in the slave > cluster, you might see the cell having one of the three possible values 1, 2, > 3, which results in data inconsistency issue between the master and slave > clusters. > Root Cause Analysis: > In HBaseInterClusterReplicationEndpoint.createBatches() of branch-1.3, the > WAL entries from the same region could be split into different batches > according to replication RPC limit and these batches are shipped by > ReplicationSource concurrently, so the batches for the same region could > arrive at the sink in the slave clusters then apply to the region > synchronously in indeterminate order. > Solution: > In HBase 3.0.0 and 2.1.0, [~Apache9]&[~openinx]&[~fenghh] provided Serial > Replication HBASE-20046 which guarantees the order of pushing logs to slave > clusters is same as the order of requests from client in the master cluster. > It contains mainly two changes: > # Recording the replication "barriers" in ZooKeeper to synchronize the > replication across old/failed RS and new RS to provide strict ordering > semantics even in the presence of region-move or RS failure. > # Make sure the batches within one region are shipped to the slave clusters > in order. > The second part of change is exactly what we need and the minimal change to > fix the issue in this JIRA. > To fix the issue in this JIRA, we have two options: > # Cherry-Pick HBASE-20046 to branch 1.3. Pros: It also fixes the data > inconsistency issue when there is region-move or RS failure and help to > reduce the noises in our cross-cluster replication/backup validation which is > our ultimate goal. Cons: the change is big and I'm not sure for now whether > the change is self-contained or it has other dependencies which need to port > to branch 1.3 too; and we need longer time to validate and stabilize. > # Port the minimal change or make the equivalent change as the second part > of HBASE-20046 to make sure the batches within one region are shipped to the > slave clusters in order." > With limited knowledge about HBase Release Schedule and Process, I prefer > option 2 because of cons of option 1, but I'm open to option 1 and other > options. Thoughts? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23143) Region Server Crash due to 2 cells out of order ( between 2 DELETEs)
[
https://issues.apache.org/jira/browse/HBASE-23143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-23143:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Region Server Crash due to 2 cells out of order ( between 2 DELETEs)
>
>
> Key: HBASE-23143
> URL: https://issues.apache.org/jira/browse/HBASE-23143
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Xu Cang
>Priority: Major
> Fix For: 1.8.0
>
>
> Region Server Crash due to 2 cells out of order ( between 2 DELETEs)
>
> Caused by: java.io.IOException: Added a key not lexically larger than
> previous.
> Current cell =
> 00D7F00xxQ10D52v8UY6yV0057F00bPaGT\x00057F00bPaG/0:TABLE1_ID/*1570095189597*/DeleteColumn/vlen=0/seqid=*2128373*,
>
> lastCell =
> 00D7F00xxQ10D52v8UY6yV0057F00bPaGT\x00057F00bPaG/0:TABLE1_ID/*1570095165147*/DeleteColumn/vlen=0/seqid=*2128378*
>
>
> I am aware of this JIRA: https://issues.apache.org/jira/browse/HBASE-22862
> Though it's slightly different, HBASE-22862 issue was caused One Delete and
> One Put.
> This issue I am reporting is caused by 2 Deletes
>
> Has anyone seen this issue?
>
> After I read the code and debugged the test cases.
> In AbstractHFileWriter.java
> {code:java}
> int keyComp = comparator.compareOnlyKeyPortion(lastCell, cell);{code}
> This call will always ignore SequenceId. And time stamps are in the correct
> order (above case)
> And since these 2 cells have same KEY. The comparison result should be 0.
> *only possible issue I can think of is, in this code piece: in
> CellComparator.java:*
> {code:java}
> Bytes.compareTo(left.getRowArray(), left.getRowOffset(), left.getRowLength(),
> right.getRowArray(), right.getRowOffset(), right.getRowLength());{code}
> The getRowLength() returns a wrong value.
> Or the offset is messed up. (?)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23240) branch-1 master and regionservers do not start when compiled against Hadoop 3.2.1
[ https://issues.apache.org/jira/browse/HBASE-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-23240: -- Fix Version/s: (was: 1.7.0) 1.8.0 > branch-1 master and regionservers do not start when compiled against Hadoop > 3.2.1 > - > > Key: HBASE-23240 > URL: https://issues.apache.org/jira/browse/HBASE-23240 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.5.0 >Reporter: Lars Hofhansl >Priority: Major > Fix For: 1.8.0 > > > Exception in thread "main" java.lang.NoSuchMethodError: > com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357) > at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338) > at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679) > at > org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:339) > at > org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:572) > at > org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:174) > at > org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:156) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23147) Branches-1 shaded artifact for mapreduce integration misses MainClass
[
https://issues.apache.org/jira/browse/HBASE-23147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-23147:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Branches-1 shaded artifact for mapreduce integration misses MainClass
> -
>
> Key: HBASE-23147
> URL: https://issues.apache.org/jira/browse/HBASE-23147
> Project: HBase
> Issue Type: Bug
> Components: Client, mapreduce
>Affects Versions: 1.5.0, 1.3.6, 1.4.11
>Reporter: Sean Busbey
>Priority: Major
> Labels: beginner
> Fix For: 1.8.0
>
>
> the shaded artifact we intend for folks to use when doing mapreduce stuff in
> branches-1 is {{hbase-shaded-server}}, but it fails to define the same
> {{MainClass}} for the jar as the {{hbase-server}} artifact. This prevents
> commands like this from working:
> {code}
> $ hadoop jar some/path/to/hbase-shaded-server-1.4.11-SNAPSHOT.jar importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars $(hbase mapredcp)
> Exception in thread "main" java.lang.ClassNotFoundException: importtsv
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:232)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {code}
> The {{hbase-shaded-server}} module pom should have the same maven-jar-plugin
> config for setting the main class as the {{hbase-server}} module.
> This is not an issue in branches-2+ because as a part of moving this stuff
> into a {{hbase-mapreduce}} and {{hbase-shaded-mapreduce}} (HBASE-18697) we
> corrected this gap on the shaded artifact.
> Work around by specifying the class manually
> {code}
> hadoop jar some/path/to/hbase-shaded-server-1.4.11-SNAPSHOT.jar
> org.apache.hadoop.hbase.mapreduce.Driver importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars $(hbase mapredcp)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23602) TTL Before Which No Data is Purged
[ https://issues.apache.org/jira/browse/HBASE-23602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-23602: -- Fix Version/s: (was: 1.7.0) 1.8.0 > TTL Before Which No Data is Purged > -- > > Key: HBASE-23602 > URL: https://issues.apache.org/jira/browse/HBASE-23602 > Project: HBase > Issue Type: New Feature >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0 > > > HBase currently offers operators a choice. They can set > KEEP_DELETED_CELLS=true and VERSIONS to max value, plus no TTL, and they will > always have a complete history of all changes (but high storage costs and > penalties to read performance). Or they can have KEEP_DELETED_CELLS=false and > VERSIONS/TTL set to some reasonable values, but that means that major > compactions can destroy the ability to do a consistent snapshot read of any > prior time. (This limits the usefulness and correctness of, for example, > Phoenix's SCN lookback feature.) > I propose having a new TTL property to give a minimum age that an expired or > deleted Cell would have to achieve before it could be purged. (I see that > HBASE-10118 already does something similar for the delete markers > themselves.) > This would allow operators to have a consistent history for some finite > amount of recent time while still purging out the "long tail" of obsolete / > deleted versions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23639) Moving classes out of hbase-it /test for direct API use of chaos.
[ https://issues.apache.org/jira/browse/HBASE-23639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-23639: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Moving classes out of hbase-it /test for direct API use of chaos. > - > > Key: HBASE-23639 > URL: https://issues.apache.org/jira/browse/HBASE-23639 > Project: HBase > Issue Type: Improvement > Components: API, integration tests, test >Reporter: Mihir Monani >Assignee: Lokesh Khurana >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0 > > Attachments: HBASE-23639-master.patch > > > In hbase-it package, Class like RESTApiClusterManager and HBaseClusterManager > which has some part of implementation code for Chaos Action. > > If any user wants to create private implementation which resembles > HBaseClusterManager, then it has to be merged with hbase-it package as > ClusterManager Interface and it's extended implementations are private. > > We should make them public so anyone can have their own implementation and > need to be merged with hbase-it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23768) Backport HBASE-17115 (HMaster/HRegion Info Server does not honour admin.acl) to 1.x
[ https://issues.apache.org/jira/browse/HBASE-23768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-23768: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Backport HBASE-17115 (HMaster/HRegion Info Server does not honour admin.acl) > to 1.x > --- > > Key: HBASE-23768 > URL: https://issues.apache.org/jira/browse/HBASE-23768 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25024) [Flake Test][branch-1] TestClientOperationInterrupt#testInterrupt50Percent
[
https://issues.apache.org/jira/browse/HBASE-25024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25024:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> [Flake Test][branch-1] TestClientOperationInterrupt#testInterrupt50Percent
> --
>
> Key: HBASE-25024
> URL: https://issues.apache.org/jira/browse/HBASE-25024
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Priority: Minor
> Fix For: 1.8.0
>
>
> Sometimes threads could finish faster before interrupt() gets called.
> {code}
> // ...
> t.start();
> }
> int expectedNoExNum = nbThread / 2;
> for (int i = 0; i < nbThread / 2; i++) {
> if (threads.get(i).getState().equals(Thread.State.TERMINATED)) {
> expectedNoExNum--;
> }
> threads.get(i).interrupt();
> }
> {code}
> So this test could get failed sometimes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23948) Backport HBASE-23146 (Support CheckAndMutate with multiple conditions) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-23948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-23948: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Backport HBASE-23146 (Support CheckAndMutate with multiple conditions) to > branch-1 > -- > > Key: HBASE-23948 > URL: https://issues.apache.org/jira/browse/HBASE-23948 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Priority: Major > Fix For: 1.8.0 > > > Backport HBASE-23146 (Support CheckAndMutate with multiple conditions) to > branch-1, including updates to REST (HBASE-23924) and Thrift (HBASE-23925). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25019) check if we're impacted by MASSEMBLY-941 and mitigate if needed
[ https://issues.apache.org/jira/browse/HBASE-25019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25019: -- Fix Version/s: (was: 1.7.0) 1.8.0 > check if we're impacted by MASSEMBLY-941 and mitigate if needed > --- > > Key: HBASE-25019 > URL: https://issues.apache.org/jira/browse/HBASE-25019 > Project: HBase > Issue Type: Task > Components: scripts >Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.3.1, 1.6.0, 1.7.0 >Reporter: Sean Busbey >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0, 2.3.6 > > > MASSEMBLY-941 notes a bug starting in version 3.2.0 of the assembly plugin > where scripts lose their executable bit. > We've had this version since updating to the apache parent pom version 22 in > HBASE-23675. We should check our release artifacts to see if we are impacted > and if so downgrade the assembly plugin in our poms to 3.1.1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25510) Optimize TableName.valueOf from O(n) to O(1). We can get benefits when the number of tables in the cluster is greater than dozens
[
https://issues.apache.org/jira/browse/HBASE-25510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25510:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Optimize TableName.valueOf from O(n) to O(1). We can get benefits when the
> number of tables in the cluster is greater than dozens
> --
>
> Key: HBASE-25510
> URL: https://issues.apache.org/jira/browse/HBASE-25510
> Project: HBase
> Issue Type: Improvement
> Components: master, Replication
>Affects Versions: 1.2.12, 1.4.13, 2.4.1
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0
>
> Attachments: TestTableNameJMH.java, optimiz_benchmark,
> optimiz_benchmark_fix, origin_benchmark, stucks-profile-info
>
>
> Now, TableName.valueOf will try to find TableName Object in cache
> linearly(code show as below). So it is too slow when we has thousands of
> tables on cluster.
> {code:java}
> // code placeholder
> for (TableName tn : tableCache) {
> if (Bytes.equals(tn.getQualifier(), qns) && Bytes.equals(tn.getNamespace(),
> bns)) {
> return tn;
> }
> }{code}
> I try to store the object in the hash table, so it can look up more quickly.
> code like this
> {code:java}
> // code placeholder
> TableName oldTable = tableCache.get(nameAsStr);{code}
>
> In our cluster which has tens thousands of tables. (Most of that is KYLIN
> table).
> We found that in the following two cases, the TableName.valueOf method will
> severely restrict our performance.
>
> Common premise: tens of thousands table in cluster
> cause: TableName.valueOf with low performance. (because we need to traverse
> all caches linearly)
>
> Case1. Replication
> premise1: one of table write with high qps, small value, Non-batch request.
> cause too much wal entry
> premise2: deserialize WAL Entry includes calling the TableName.valueOf method.
> Cause: Replicat Stuck. A lot of WAL files pile up.
>
> Case2. Active Master Start up
> NamespaceStateManager init should init all RegionInfo, and regioninfo init
> will call TableName.valueOf. It will cost some time if TableName.valueOf is
> slow.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25624) Bound LoadBalancer's RegionLocationFinder cache
[ https://issues.apache.org/jira/browse/HBASE-25624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25624: -- Fix Version/s: (was: 1.7.0) 1.8.0 > Bound LoadBalancer's RegionLocationFinder cache > --- > > Key: HBASE-25624 > URL: https://issues.apache.org/jira/browse/HBASE-25624 > Project: HBase > Issue Type: Bug > Components: Balancer, master, Operability >Affects Versions: 1.6.0, 2.4.1 >Reporter: Andrew Kyle Purtell >Assignee: Prathyusha >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 1.8.0, 2.4.3 > > > We have a large table in production that causes the balancer's > RegionLocationFinder cache to consume 4 GB of heap, which, among other > factors, triggered OOMEs, and made us aware of this problem. > RegionLocationFinder embeds a cache backed by Guava's CacheLoader. The > RegionLocationFinder cache comes to consume heap for RegionInfos for all > table regions and all HDFS block locations of all store files for all regions > of all tables. > The only limit we pass to the CacheBuilder is an expiration time of 1440 > milliseconds for individual cache entries. That's 4 hours. That's much too > long; however, the cache also periodically refreshes itself, where the need > for a refresh is checked whenever BaseLoadBalancer calls > RegionLocationFinder's setClusterMetrics() method, which defeats the > expiration based limit anyway. > We should be bounding this cache with effective resource controls. Time based > expiry is fine but the periodic refresh logic must be removed to make it > effective. Implement size based limits too. CacheBuilder#maximumSize will > limit by number cache entries. This might be fine but > CacheBuilder#maximumWeight would be better, where weight is something > determined by the API user. In this case it can be an estimate of the heap > size of the hash map entries kept in the cache. > Default should remain unbounded. Specific bounds should be supported by new > site configuration options. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25217) [Metrics] Add metrics for Call in IPC response queue
[ https://issues.apache.org/jira/browse/HBASE-25217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25217: -- Fix Version/s: (was: 1.7.0) 1.8.0 > [Metrics] Add metrics for Call in IPC response queue > > > Key: HBASE-25217 > URL: https://issues.apache.org/jira/browse/HBASE-25217 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.8.0 > > > Add metrics for response queue. > E.g., number of Call/RpcResponse in queue, size of Call/RpcResponse in queue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25664) Adding replication peer should handle the undeleted queue exception
[
https://issues.apache.org/jira/browse/HBASE-25664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25664:
--
Fix Version/s: (was: 1.7.0)
1.8.0
> Adding replication peer should handle the undeleted queue exception
> ---
>
> Key: HBASE-25664
> URL: https://issues.apache.org/jira/browse/HBASE-25664
> Project: HBase
> Issue Type: Improvement
>Reporter: Sandeep Pal
>Assignee: Sandeep Pal
>Priority: Major
> Fix For: 1.8.0
>
>
> Currently, if we try to add a peer and there is a replication queue existing
> for that peer, it doesn't let the replication peer created.
> Instead, we should delete the queue and proceed with the creating of
> replication peer. Any queue without no corresponding replication peer is
> useless anyway. So, we shouldn't wait for cleaner to come and clean it before
> creating the peer.
>
> {code:java}
> org.apache.hadoop.hbase.replication.ReplicationException: undeleted queue for
> peerId: xyz_peerid, replicator: hostname.fakeaddress.com,60020,1607576586258,
> queueId: xyz_peerid
> java.lang.RuntimeException:
> org.apache.hadoop.hbase.replication.ReplicationException: undeleted queue for
> peerId: xyz_peerid, replicator: hostname.fakeaddress.com,60020,1607576586258,
> queueId: xyz_peerid
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25725) Release 1.7.0
Reid Chan created HBASE-25725: - Summary: Release 1.7.0 Key: HBASE-25725 URL: https://issues.apache.org/jira/browse/HBASE-25725 Project: HBase Issue Type: Task Reporter: Reid Chan Assignee: Reid Chan Discussion is [here|https://lists.apache.org/thread.html/r34d3fb86d667f8b3e58cbba78655733ac76e10f5883650f4910adc5c%40%3Cuser.hbase.apache.org%3E], this may or may not be the last release of branch-1. But please allow me to go through the whole process before making any words. I'll spend the upcoming next whole week to do this release, having said to my employer. cc [~andrew.purt...@gmail.com] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25677) Server+table counters on each scan #nextRaw invocation becomes a bottleneck when heavy load
[
https://issues.apache.org/jira/browse/HBASE-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25677:
--
Fix Version/s: 1.7.0
> Server+table counters on each scan #nextRaw invocation becomes a bottleneck
> when heavy load
> ---
>
> Key: HBASE-25677
> URL: https://issues.apache.org/jira/browse/HBASE-25677
> Project: HBase
> Issue Type: Sub-task
> Components: metrics
>Affects Versions: 2.3.2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.3
>
>
> On a heavily loaded server mostly doing reads/scan, I saw that 90+% of
> handlers were BLOCKED in this fashion in thread dumps:
> {code}
> "RpcServer.default.FPBQ.Fifo.handler=117,queue=17,port=16020" #161 daemon
> prio=5 os_prio=0 tid=0x7f748757f000 nid=0x73e9 waiting for monitor entry
> [0x7f74783e]
> java.lang.Thread.State: BLOCKED (on object monitor)
>at
> java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1674)
>- waiting to lock <0x7f7647e3cc38> (a
> java.util.concurrent.ConcurrentHashMap$Node)
>at
> org.apache.hadoop.hbase.regionserver.MetricsTableQueryMeterImpl.getOrCreateTableMeter(MetricsTableQueryMeterImpl.java:80)
>at
> org.apache.hadoop.hbase.regionserver.MetricsTableQueryMeterImpl.updateTableReadQueryMeter(MetricsTableQueryMeterImpl.java:90)
>at
> org.apache.hadoop.hbase.regionserver.RegionServerTableMetrics.updateTableReadQueryMeter(RegionServerTableMetrics.java:89)
>at
> org.apache.hadoop.hbase.regionserver.MetricsRegionServer.updateReadQueryMeter(MetricsRegionServer.java:274)
>at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:6742)
>at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3319)
>- locked <0x7f896c0165a0> (a
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl)
>at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3566)
>at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44858)
>at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:393)
>at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
>at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
>at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> {code}
> It kept up for good periods of time.
> I saw it to a leser extent on other servers, with less load.
> These RS had 400+ Regions a good few of which were serving out scan reads;
> the server was doing ~1M hits a second. In this scenario, I saw the above
> bottleneck.
> Looking at it, it came in w/ when the parent issue feature was added. There
> are these read counts and then there were also write counts. The write counts
> are mostly batch-based. Let me do same thing here for the read update the
> central server+table count after scan is done rather than per invocation of
> #nextRaw.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-21674) Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from thrift1 server) to branch-1
[
https://issues.apache.org/jira/browse/HBASE-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285606#comment-17285606
]
Reid Chan commented on HBASE-21674:
---
ping [~apurtell], [~busbey]
could you spare some cycles to take a look at PR:
https://github.com/apache/hbase/pull/2941
{quote}major version 52 is newer than 51, the highest major version supported
by this compiler. It is recommended that the compiler be upgraded.
{quote}
The warning seems about JDK complier 1.7, is it ok to ignore? I'm not sure
about that.
> Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from
> thrift1 server) to branch-1
>
>
> Key: HBASE-21674
> URL: https://issues.apache.org/jira/browse/HBASE-21674
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-21674) Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from thrift1 server) to branch-1
[ https://issues.apache.org/jira/browse/HBASE-21674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-21674: -- Fix Version/s: 1.7.0 > Port HBASE-21652 (Refactor ThriftServer making thrift2 server inherited from > thrift1 server) to branch-1 > > > Key: HBASE-21674 > URL: https://issues.apache.org/jira/browse/HBASE-21674 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Kyle Purtell >Assignee: Yutong Xiao >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25543) When configuration "hadoop.security.authorization" is set to false, the system will still try to authorize an RPC and raise AccessDeniedException
[
https://issues.apache.org/jira/browse/HBASE-25543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17279429#comment-17279429
]
Reid Chan commented on HBASE-25543:
---
Nice catch [~xytss123] , please also provide PRs for branch-1 and branch-2.
> When configuration "hadoop.security.authorization" is set to false, the
> system will still try to authorize an RPC and raise AccessDeniedException
> --
>
> Key: HBASE-25543
> URL: https://issues.apache.org/jira/browse/HBASE-25543
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Minor
>
> In method processOneRpc(Bytebuffer buf) in RpcServer.java (branch-1),
> ServerRpcConnection.java (branch-2, master), if connectionHeadRead is set to
> false, the method authorizeConnection() will be invoked whatever the boolean
> authorize is true or false.
> {code:java}
> if (!authorizeConnection()) {
> // Throw FatalConnectionException wrapping ACE so client does right thing
> and closes
> // down the connection instead of trying to read non-existent retun.
> throw new AccessDeniedException("Connection from " + this + " for service "
> +
> connectionHeader.getServiceName() + " is unauthorized for user: " + ugi);
> }
> {code}
> In method authorizeConnection()
> {code:java}
> if (ugi != null && ugi.getRealUser() != null
> && (authMethod != AuthMethod.DIGEST)) {
> ProxyUsers.authorize(ugi, this.getHostAddress(), conf);
> }{code}
> ProxyUsers.authorize() will raise AuthorizationException.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17274125#comment-17274125
]
Reid Chan commented on HBASE-20993:
---
It requires some changes on wire protocol, not that easy to satisify backward
compatibility.
And I don't have that env to test it anymore (some changes in work & life), so
it hung.
[~0x62ash] feel free to take it, i unassigned myself.
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 1.2.6, 1.3.2, 1.2.7, 1.4.7
>Reporter: Reid Chan
>Priority: Critical
> Fix For: 1.7.0
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.010.patch, HBASE-20993.branch-1.011.patch,
> HBASE-20993.branch-1.012.patch, HBASE-20993.branch-1.013.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
>
[jira] [Assigned] (HBASE-20993) [Auth] IPC client fallback to simple auth allowed doesn't work
[
https://issues.apache.org/jira/browse/HBASE-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan reassigned HBASE-20993:
-
Assignee: (was: Reid Chan)
> [Auth] IPC client fallback to simple auth allowed doesn't work
> --
>
> Key: HBASE-20993
> URL: https://issues.apache.org/jira/browse/HBASE-20993
> Project: HBase
> Issue Type: Bug
> Components: Client, IPC/RPC, security
>Affects Versions: 1.2.6, 1.3.2, 1.2.7, 1.4.7
>Reporter: Reid Chan
>Priority: Critical
> Fix For: 1.7.0
>
> Attachments: HBASE-20993.001.patch,
> HBASE-20993.003.branch-1.flowchart.png, HBASE-20993.branch-1.002.patch,
> HBASE-20993.branch-1.003.patch, HBASE-20993.branch-1.004.patch,
> HBASE-20993.branch-1.005.patch, HBASE-20993.branch-1.006.patch,
> HBASE-20993.branch-1.007.patch, HBASE-20993.branch-1.008.patch,
> HBASE-20993.branch-1.009.patch, HBASE-20993.branch-1.009.patch,
> HBASE-20993.branch-1.010.patch, HBASE-20993.branch-1.011.patch,
> HBASE-20993.branch-1.012.patch, HBASE-20993.branch-1.013.patch,
> HBASE-20993.branch-1.2.001.patch, HBASE-20993.branch-1.wip.002.patch,
> HBASE-20993.branch-1.wip.patch, yetus-local-testpatch-output-009.txt
>
>
> It is easily reproducible.
> client's hbase-site.xml: hadoop.security.authentication:kerberos,
> hbase.security.authentication:kerberos,
> hbase.ipc.client.fallback-to-simple-auth-allowed:true, keytab and principal
> are right set
> A simple auth hbase cluster, a kerberized hbase client application.
> application trying to r/w/c/d table will have following exception:
> {code}
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:617)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$700(RpcClientImpl.java:162)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:743)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:740)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:740)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:906)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:873)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1241)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58383)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1592)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1530)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1552)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1581)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1738)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4297)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4289)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsyncV2(HBaseAdmin.java:753)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:674)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:607)
> at
>
[jira] [Commented] (HBASE-25308) [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous
[ https://issues.apache.org/jira/browse/HBASE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242866#comment-17242866 ] Reid Chan commented on HBASE-25308: --- Got it, thanks for clarification. > [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous > - > > Key: HBASE-25308 > URL: https://issues.apache.org/jira/browse/HBASE-25308 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 1.7.0 > > > We are again having classpath versioning issues related to Guava in our > branch-1 based application. > Hadoop 3, HBase 2, Phoenix 5, and other projects deal with Guava > cross-version incompatibilities, as they manifest on a combined classpath > with other components, via shading. > I propose to do a global search and replace of all direct uses of Guava in > our branch-1 code base and refer to Guava as provided in hbase-thirdparty's > hbase-shaded-miscellaneous. This will protect HBase branch-1 from Guava > cross-version vagaries just like the same technique protects branch-2 and > branch-2 based releases. > There are a couple of Public or LimitedPrivate interfaces that incorporate > Guava's HostAndPort and Service that will be indirectly impacted. We are > about to release a new minor branch-1 version, 1.7.0, and this would be a > great opportunity to introduce this kind of change in a manner consistent > with semantic versioning and our compatibility policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25308) [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous
[ https://issues.apache.org/jira/browse/HBASE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242019#comment-17242019 ] Reid Chan commented on HBASE-25308: --- Leave alone checkstyle, there's *COMPILATION ERROR* > [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous > - > > Key: HBASE-25308 > URL: https://issues.apache.org/jira/browse/HBASE-25308 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 1.7.0 > > > We are again having classpath versioning issues related to Guava in our > branch-1 based application. > Hadoop 3, HBase 2, Phoenix 5, and other projects deal with Guava > cross-version incompatibilities, as they manifest on a combined classpath > with other components, via shading. > I propose to do a global search and replace of all direct uses of Guava in > our branch-1 code base and refer to Guava as provided in hbase-thirdparty's > hbase-shaded-miscellaneous. This will protect HBase branch-1 from Guava > cross-version vagaries just like the same technique protects branch-2 and > branch-2 based releases. > There are a couple of Public or LimitedPrivate interfaces that incorporate > Guava's HostAndPort and Service that will be indirectly impacted. We are > about to release a new minor branch-1 version, 1.7.0, and this would be a > great opportunity to introduce this kind of change in a manner consistent > with semantic versioning and our compatibility policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25308) [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous
[ https://issues.apache.org/jira/browse/HBASE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242018#comment-17242018 ] Reid Chan commented on HBASE-25308: --- do we need to take care those (x)? Or we can just directly merge > [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous > - > > Key: HBASE-25308 > URL: https://issues.apache.org/jira/browse/HBASE-25308 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 1.7.0 > > > We are again having classpath versioning issues related to Guava in our > branch-1 based application. > Hadoop 3, HBase 2, Phoenix 5, and other projects deal with Guava > cross-version incompatibilities, as they manifest on a combined classpath > with other components, via shading. > I propose to do a global search and replace of all direct uses of Guava in > our branch-1 code base and refer to Guava as provided in hbase-thirdparty's > hbase-shaded-miscellaneous. This will protect HBase branch-1 from Guava > cross-version vagaries just like the same technique protects branch-2 and > branch-2 based releases. > There are a couple of Public or LimitedPrivate interfaces that incorporate > Guava's HostAndPort and Service that will be indirectly impacted. We are > about to release a new minor branch-1 version, 1.7.0, and this would be a > great opportunity to introduce this kind of change in a manner consistent > with semantic versioning and our compatibility policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25308) [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous
[ https://issues.apache.org/jira/browse/HBASE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242010#comment-17242010 ] Reid Chan commented on HBASE-25308: --- Looked at the PR, many codes need to be polished > [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous > - > > Key: HBASE-25308 > URL: https://issues.apache.org/jira/browse/HBASE-25308 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 1.7.0 > > > We are again having classpath versioning issues related to Guava in our > branch-1 based application. > Hadoop 3, HBase 2, Phoenix 5, and other projects deal with Guava > cross-version incompatibilities, as they manifest on a combined classpath > with other components, via shading. > I propose to do a global search and replace of all direct uses of Guava in > our branch-1 code base and refer to Guava as provided in hbase-thirdparty's > hbase-shaded-miscellaneous. This will protect HBase branch-1 from Guava > cross-version vagaries just like the same technique protects branch-2 and > branch-2 based releases. > There are a couple of Public or LimitedPrivate interfaces that incorporate > Guava's HostAndPort and Service that will be indirectly impacted. We are > about to release a new minor branch-1 version, 1.7.0, and this would be a > great opportunity to introduce this kind of change in a manner consistent > with semantic versioning and our compatibility policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25285) [branch-1] Fix two methods removed problems reported by compat-check
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan resolved HBASE-25285.
---
Fix Version/s: 1.7.0
Hadoop Flags: Reviewed
Resolution: Fixed
> [branch-1] Fix two methods removed problems reported by compat-check
>
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 1.7.0
>
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25285) [branch-1] Fix two methods removed problems reported by compat-check
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25285:
--
Component/s: API
> [branch-1] Fix two methods removed problems reported by compat-check
>
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
> Components: API
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 1.7.0
>
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25314) branch-1 docker mode for yetus fails
[
https://issues.apache.org/jira/browse/HBASE-25314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17236549#comment-17236549
]
Reid Chan commented on HBASE-25314:
---
HBASE-25313 filed, waiting for review
> branch-1 docker mode for yetus fails
>
>
> Key: HBASE-25314
> URL: https://issues.apache.org/jira/browse/HBASE-25314
> Project: HBase
> Issue Type: Bug
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
>
> {noformat}
> 15:30:41 Step 28/33 : RUN gem install rubocop:'<= 0.81'
> 15:30:41 ---> Running in 21103fb7944c
> 15:30:42 Building native extensions. This could take a while...
> 15:30:43 [91mERROR: Error installing rubocop:
> 15:30:43 parallel requires Ruby version >= 2.5.
> 15:30:43 [0mSuccessfully installed jaro_winkler-1.5.4
> 15:30:44 The command '/bin/sh -c gem install rubocop:'<= 0.81'' returned a
> non-zero code: 1
> 15:30:44 ERROR: Docker failed to build yetus/hbase:b249092a5f.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25313) [branch-1] Fix the broken pre-commit
[
https://issues.apache.org/jira/browse/HBASE-25313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25313:
--
Description:
{code}
Archive JUnit-formatted test results
[2020-11-20T02:58:50.501Z] Recording test results
[2020-11-20T02:58:50.641Z] No test report files were found. Configuration error?
{code}
branch-1's pre-commit keeps failing with error above, we need to fix it.
> [branch-1] Fix the broken pre-commit
>
>
> Key: HBASE-25313
> URL: https://issues.apache.org/jira/browse/HBASE-25313
> Project: HBase
> Issue Type: Bug
> Components: jenkins
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
> Fix For: 1.7.0
>
>
> {code}
> Archive JUnit-formatted test results
> [2020-11-20T02:58:50.501Z] Recording test results
> [2020-11-20T02:58:50.641Z] No test report files were found. Configuration
> error?
> {code}
> branch-1's pre-commit keeps failing with error above, we need to fix it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25313) [branch-1] Fix the broken pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25313: -- Component/s: jenkins > [branch-1] Fix the broken pre-commit > > > Key: HBASE-25313 > URL: https://issues.apache.org/jira/browse/HBASE-25313 > Project: HBase > Issue Type: Bug > Components: jenkins >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25313) [branch-1] Fix the broken pre-commit
[ https://issues.apache.org/jira/browse/HBASE-25313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25313: -- Fix Version/s: 1.7.0 > [branch-1] Fix the broken pre-commit > > > Key: HBASE-25313 > URL: https://issues.apache.org/jira/browse/HBASE-25313 > Project: HBase > Issue Type: Bug >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25313) [branch-1] Fix the broken pre-commit
Reid Chan created HBASE-25313: - Summary: [branch-1] Fix the broken pre-commit Key: HBASE-25313 URL: https://issues.apache.org/jira/browse/HBASE-25313 Project: HBase Issue Type: Bug Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25308) [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous
[ https://issues.apache.org/jira/browse/HBASE-25308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235846#comment-17235846 ] Reid Chan commented on HBASE-25308: --- Let's do that. > [branch-1] Consume Guava from hbase-thirdparty hbase-shaded-miscellaneous > - > > Key: HBASE-25308 > URL: https://issues.apache.org/jira/browse/HBASE-25308 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 1.7.0 > > > We are again having classpath versioning issues related to Guava in our > branch-1 based application. > Hadoop 3, HBase 2, Phoenix 5, and other projects deal with Guava > cross-version incompatibilities, as they manifest on a combined classpath > with other components, via shading. > I propose to do a global search and replace of all direct uses of Guava in > our branch-1 code base and refer to Guava as provided in hbase-thirdparty's > hbase-shaded-miscellaneous. This will protect HBase branch-1 from Guava > cross-version vagaries just like the same technique protects branch-2 and > branch-2 based releases. > There are a couple of Public or LimitedPrivate interfaces that incorporate > Guava's HostAndPort and Service that will be indirectly impacted. We are > about to release a new minor branch-1 version, 1.7.0, and this would be a > great opportunity to introduce this kind of change in a manner consistent > with semantic versioning and our compatibility policies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25285) [branch-1] Fix two methods removed problems reported by compat-check
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232572#comment-17232572
]
Reid Chan commented on HBASE-25285:
---
Because they are HBaseInterfaceAudience.COPROC.
Let's keep them, just in case any coprocessor uses.
> [branch-1] Fix two methods removed problems reported by compat-check
>
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25285) [branch-1] Fix two methods removed problems reported by compat-check
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25285:
--
Summary: [branch-1] Fix two methods removed problems reported by
compat-check (was: [branch-1] Fix two methods removed problems from
compat-check reports)
> [branch-1] Fix two methods removed problems reported by compat-check
>
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25285) [branch-1] Fix two methods removed problems from compat-check reports
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25285:
--
Summary: [branch-1] Fix two methods removed problems from compat-check
reports (was: [branch-1] Fix problems from compat-check reports)
> [branch-1] Fix two methods removed problems from compat-check reports
> -
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25285) [branch-1] Fix problems from compat-check reports
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232528#comment-17232528
]
Reid Chan commented on HBASE-25285:
---
They were introduced by HBASE-24174 and HBASE-23631
> [branch-1] Fix problems from compat-check reports
> -
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25285) [branch-1] Fix problems from compat-check reports
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25285:
--
Summary: [branch-1] Fix problems from compat-check reports (was:
[branch-1] Fix problems from compat-check reports=)
> [branch-1] Fix problems from compat-check reports
> -
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25285) [branch-1] Fix problems from compat-check reports
[
https://issues.apache.org/jira/browse/HBASE-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232224#comment-17232224
]
Reid Chan commented on HBASE-25285:
---
cc [~andrew.purt...@gmail.com]
> [branch-1] Fix problems from compat-check reports
> -
>
> Key: HBASE-25285
> URL: https://issues.apache.org/jira/browse/HBASE-25285
> Project: HBase
> Issue Type: Bug
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Major
>
> {code}
> hbase-server-1.6.0.jar, RpcServerInterface.class
> package org.apache.hadoop.hbase.ipc
> RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] :
> void
> hbase-server-1.6.0.jar, Store.class
> package org.apache.hadoop.hbase.regionserver
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
> Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
> boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
> StoreFile.Writer
> {code}
> These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25285) [branch-1] Fix problems from compat-check reports=
Reid Chan created HBASE-25285:
-
Summary: [branch-1] Fix problems from compat-check reports=
Key: HBASE-25285
URL: https://issues.apache.org/jira/browse/HBASE-25285
Project: HBase
Issue Type: Bug
Reporter: Reid Chan
Assignee: Reid Chan
{code}
hbase-server-1.6.0.jar, RpcServerInterface.class
package org.apache.hadoop.hbase.ipc
RpcServerInterface.refreshAuthManager ( PolicyProvider p1 ) [abstract] : void
hbase-server-1.6.0.jar, Store.class
package org.apache.hadoop.hbase.regionserver
Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
boolean p4, boolean p5, boolean p6 ) [abstract] : StoreFile.Writer
Store.createWriterInTmp ( long p1, Compression.Algorithm p2, boolean p3,
boolean p4, boolean p5, boolean p6, TimeRangeTracker p7 ) [abstract] :
StoreFile.Writer
{code}
These two removed methods may be problematic, let's review them again.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25217) [Metrics] Add metrics for Call in IPC response queue
[ https://issues.apache.org/jira/browse/HBASE-25217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25217: -- Fix Version/s: 1.7.0 3.0.0-alpha-1 > [Metrics] Add metrics for Call in IPC response queue > > > Key: HBASE-25217 > URL: https://issues.apache.org/jira/browse/HBASE-25217 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0 > > > Add metrics for response queue. > E.g., number of Call/RpcResponse in queue, size of Call/RpcResponse in queue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25217) [Metrics] Add metrics for Call in IPC response queue
Reid Chan created HBASE-25217: - Summary: [Metrics] Add metrics for Call in IPC response queue Key: HBASE-25217 URL: https://issues.apache.org/jira/browse/HBASE-25217 Project: HBase Issue Type: Improvement Components: metrics Reporter: Reid Chan Assignee: Reid Chan Add metrics for response queue. E.g., number of Call/RpcResponse in queue, size of Call/RpcResponse in queue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[ https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25189. --- Hadoop Flags: Reviewed Resolution: Fixed > [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level > --- > > Key: HBASE-25189 > URL: https://issues.apache.org/jira/browse/HBASE-25189 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7 > > > Add *checkAndPut* and *checkAndDelete* table latency metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[ https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25189: -- Description: Add *checkAndPut* and *checkAndDelete* table latency metrics. > [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level > --- > > Key: HBASE-25189 > URL: https://issues.apache.org/jira/browse/HBASE-25189 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7 > > > Add *checkAndPut* and *checkAndDelete* table latency metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[ https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25189: -- Fix Version/s: 2.2.7 1.7.0 2.3.3 3.0.0-alpha-1 > [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level > --- > > Key: HBASE-25189 > URL: https://issues.apache.org/jira/browse/HBASE-25189 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[ https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25189: -- Component/s: metrics > [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level > --- > > Key: HBASE-25189 > URL: https://issues.apache.org/jira/browse/HBASE-25189 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.2.7 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
[
https://issues.apache.org/jira/browse/HBASE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan resolved HBASE-25195.
---
Hadoop Flags: Reviewed
Resolution: Fixed
> [branch-1] getNumOpenConnections is not effective
> -
>
> Key: HBASE-25195
> URL: https://issues.apache.org/jira/browse/HBASE-25195
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
> In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
[
https://issues.apache.org/jira/browse/HBASE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25195:
--
Description:
In MetricsHBaseServerWrapperImpl
{code}
@Override
public int getNumOpenConnections() {
if (!isServerStarted() || this.server.connectionList == null) {
return 0;
}
return server.connectionList.size();
}
{code}
But connectionList is created by Collections.synchronizedList(new
LinkedList());
So each connectionList.size() will compete for the mutex, we can easily replace
it with another parameter *numConnections*, as for metrics, it is enough.
> [branch-1] getNumOpenConnections is not effective
> -
>
> Key: HBASE-25195
> URL: https://issues.apache.org/jira/browse/HBASE-25195
> Project: HBase
> Issue Type: Improvement
> Components: metrics
> Environment: In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
> In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
[
https://issues.apache.org/jira/browse/HBASE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-25195:
--
Environment: (was: In MetricsHBaseServerWrapperImpl
{code}
@Override
public int getNumOpenConnections() {
if (!isServerStarted() || this.server.connectionList == null) {
return 0;
}
return server.connectionList.size();
}
{code}
But connectionList is created by Collections.synchronizedList(new
LinkedList());
So each connectionList.size() will compete for the mutex, we can easily replace
it with another parameter *numConnections*, as for metrics, it is enough.)
> [branch-1] getNumOpenConnections is not effective
> -
>
> Key: HBASE-25195
> URL: https://issues.apache.org/jira/browse/HBASE-25195
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
> In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
[
https://issues.apache.org/jira/browse/HBASE-25195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216113#comment-17216113
]
Reid Chan commented on HBASE-25195:
---
branch-2+ don't have this issue.
> [branch-1] getNumOpenConnections is not effective
> -
>
> Key: HBASE-25195
> URL: https://issues.apache.org/jira/browse/HBASE-25195
> Project: HBase
> Issue Type: Improvement
> Components: metrics
> Environment: In MetricsHBaseServerWrapperImpl
> {code}
> @Override
> public int getNumOpenConnections() {
> if (!isServerStarted() || this.server.connectionList == null) {
> return 0;
> }
> return server.connectionList.size();
> }
> {code}
> But connectionList is created by Collections.synchronizedList(new
> LinkedList());
> So each connectionList.size() will compete for the mutex, we can easily
> replace it with another parameter *numConnections*, as for metrics, it is
> enough.
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25195) [branch-1] getNumOpenConnections is not effective
Reid Chan created HBASE-25195:
-
Summary: [branch-1] getNumOpenConnections is not effective
Key: HBASE-25195
URL: https://issues.apache.org/jira/browse/HBASE-25195
Project: HBase
Issue Type: Improvement
Components: metrics
Environment: In MetricsHBaseServerWrapperImpl
{code}
@Override
public int getNumOpenConnections() {
if (!isServerStarted() || this.server.connectionList == null) {
return 0;
}
return server.connectionList.size();
}
{code}
But connectionList is created by Collections.synchronizedList(new
LinkedList());
So each connectionList.size() will compete for the mutex, we can easily replace
it with another parameter *numConnections*, as for metrics, it is enough.
Reporter: Reid Chan
Assignee: Reid Chan
Fix For: 1.7.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24849) Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-24849. --- Hadoop Flags: Reviewed Resolution: Fixed > Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when > one of the WAL needs a roll > -- > > Key: HBASE-24849 > URL: https://issues.apache.org/jira/browse/HBASE-24849 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: wenfeiyi666 >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24849) Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-24849: -- Component/s: wal > Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when > one of the WAL needs a roll > -- > > Key: HBASE-24849 > URL: https://issues.apache.org/jira/browse/HBASE-24849 > Project: HBase > Issue Type: Bug > Components: wal >Reporter: Anoop Sam John >Assignee: wenfeiyi666 >Priority: Major > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level
[ https://issues.apache.org/jira/browse/HBASE-25189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25189: -- Summary: [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level (was: [Metrics] Add checkAndPut and checkAndDelete latency metrics) > [Metrics] Add checkAndPut and checkAndDelete latency metrics at table level > --- > > Key: HBASE-25189 > URL: https://issues.apache.org/jira/browse/HBASE-25189 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25189) [Metrics] Add checkAndPut and checkAndDelete latency metrics
Reid Chan created HBASE-25189: - Summary: [Metrics] Add checkAndPut and checkAndDelete latency metrics Key: HBASE-25189 URL: https://issues.apache.org/jira/browse/HBASE-25189 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[ https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25122. --- Hadoop Flags: Reviewed Resolution: Fixed > [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory > --- > > Key: HBASE-25122 > URL: https://issues.apache.org/jira/browse/HBASE-25122 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[ https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25122: -- Component/s: test > [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory > --- > > Key: HBASE-25122 > URL: https://issues.apache.org/jira/browse/HBASE-25122 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25082) Per table WAL metrics: appendCount and appendSize
[ https://issues.apache.org/jira/browse/HBASE-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204570#comment-17204570 ] Reid Chan commented on HBASE-25082: --- Occasionally bump in, and just an ask: for the append size why don't we use SizeHistogram but Counter. The latter will accumulate the value only > Per table WAL metrics: appendCount and appendSize > - > > Key: HBASE-25082 > URL: https://issues.apache.org/jira/browse/HBASE-25082 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Labels: observability > Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0 > > > We were trying to drill down into WAL events at a table scope and figured > that appendCount and appendSize are not exposed at a table scope. This would > be a helpful metric. > I ran the WAL benchmark tool with 10 threads and didn't see any measurable > performance difference with the attached patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
Reid Chan created HBASE-25122: - Summary: [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory Key: HBASE-25122 URL: https://issues.apache.org/jira/browse/HBASE-25122 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25122) [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory
[ https://issues.apache.org/jira/browse/HBASE-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25122: -- Fix Version/s: 1.7.0 > [Flake Test][branch-1] TestExportSnapshotWithTemporaryDirectory > --- > > Key: HBASE-25122 > URL: https://issues.apache.org/jira/browse/HBASE-25122 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25114) [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
[ https://issues.apache.org/jira/browse/HBASE-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25114. --- Fix Version/s: 1.7.0 Hadoop Flags: Reviewed Resolution: Fixed > [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose > -- > > Key: HBASE-25114 > URL: https://issues.apache.org/jira/browse/HBASE-25114 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25114) [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
Reid Chan created HBASE-25114: - Summary: [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose Key: HBASE-25114 URL: https://issues.apache.org/jira/browse/HBASE-25114 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[ https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25025. --- Hadoop Flags: Reviewed Resolution: Fixed > [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp > - > > Key: HBASE-25025 > URL: https://issues.apache.org/jira/browse/HBASE-25025 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[ https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25025: -- Component/s: test > [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp > - > > Key: HBASE-25025 > URL: https://issues.apache.org/jira/browse/HBASE-25025 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[ https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25025: -- Fix Version/s: 1.7.0 > [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp > - > > Key: HBASE-25025 > URL: https://issues.apache.org/jira/browse/HBASE-25025 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-24665. --- Resolution: Fixed > MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll > -- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.3.0, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.6, 2.3.1 > > > when use multiwal, any a wal request roll, all wal will be together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203639#comment-17203639 ] Reid Chan commented on HBASE-24665: --- oh, i saw it. It is HBASE-24849 > MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll > -- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.3.0, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6 > > > when use multiwal, any a wal request roll, all wal will be together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203637#comment-17203637 ] Reid Chan commented on HBASE-24665: --- ping [~wenfeiyi666], i left a comment in branch-1 PR, shall we proceed? > MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll > -- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.3.0, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6 > > > when use multiwal, any a wal request roll, all wal will be together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24665) MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
[ https://issues.apache.org/jira/browse/HBASE-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan reopened HBASE-24665: --- I didn't see any back-porting JIRA filed but there's a branch-1 PR, so i just reopened it for finishing the back-porting > MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll > -- > > Key: HBASE-24665 > URL: https://issues.apache.org/jira/browse/HBASE-24665 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 2.3.0, 2.1.10, 1.4.14, 2.2.6 >Reporter: wenfeiyi666 >Assignee: wenfeiyi666 >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.1, 2.4.0, 2.2.6 > > > when use multiwal, any a wal request roll, all wal will be together roll. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25030) [Flaky Test] TestRestartCluster#testClusterRestart
[ https://issues.apache.org/jira/browse/HBASE-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25030: -- Component/s: test > [Flaky Test] TestRestartCluster#testClusterRestart > -- > > Key: HBASE-25030 > URL: https://issues.apache.org/jira/browse/HBASE-25030 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25031) [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
[ https://issues.apache.org/jira/browse/HBASE-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25031: -- Component/s: test > [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer > --- > > Key: HBASE-25031 > URL: https://issues.apache.org/jira/browse/HBASE-25031 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25030) [Flaky Test] TestRestartCluster#testClusterRestart
[ https://issues.apache.org/jira/browse/HBASE-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25030. --- Hadoop Flags: Reviewed Resolution: Fixed > [Flaky Test] TestRestartCluster#testClusterRestart > -- > > Key: HBASE-25030 > URL: https://issues.apache.org/jira/browse/HBASE-25030 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25031) [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
[
https://issues.apache.org/jira/browse/HBASE-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203011#comment-17203011
]
Reid Chan commented on HBASE-25031:
---
master branch has these codes:
{code}
static void restartTargetHBaseCluster(int numSlaves) throws Exception {
IOUtils.closeQuietly(htable2);
UTIL2.restartHBaseCluster(numSlaves);
// Invalidate the cached connection state
CONF2 = UTIL2.getConfiguration();
htable2 = UTIL2.getConnection().getTable(tableName);
}
{code}
No need to fix as branch-1
> [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
> ---
>
> Key: HBASE-25031
> URL: https://issues.apache.org/jira/browse/HBASE-25031
> Project: HBase
> Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25031) [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
[ https://issues.apache.org/jira/browse/HBASE-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25031. --- Hadoop Flags: Reviewed Resolution: Fixed > [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer > --- > > Key: HBASE-25031 > URL: https://issues.apache.org/jira/browse/HBASE-25031 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25031) [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
[ https://issues.apache.org/jira/browse/HBASE-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25031: -- Fix Version/s: 1.7.0 > [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer > --- > > Key: HBASE-25031 > URL: https://issues.apache.org/jira/browse/HBASE-25031 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25031) [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer
Reid Chan created HBASE-25031: - Summary: [Flaky Test] TestReplicationDisableInactivePeer#testDisableInactivePeer Key: HBASE-25031 URL: https://issues.apache.org/jira/browse/HBASE-25031 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25030) [Flaky Test] TestRestartCluster#testClusterRestart
Reid Chan created HBASE-25030: - Summary: [Flaky Test] TestRestartCluster#testClusterRestart Key: HBASE-25030 URL: https://issues.apache.org/jira/browse/HBASE-25030 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25030) [Flaky Test] TestRestartCluster#testClusterRestart
[ https://issues.apache.org/jira/browse/HBASE-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17196038#comment-17196038 ] Reid Chan commented on HBASE-25030: --- cc [~andrew.purt...@gmail.com] > [Flaky Test] TestRestartCluster#testClusterRestart > -- > > Key: HBASE-25030 > URL: https://issues.apache.org/jira/browse/HBASE-25030 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25030) [Flaky Test] TestRestartCluster#testClusterRestart
[ https://issues.apache.org/jira/browse/HBASE-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-25030: -- Fix Version/s: 1.7.0 > [Flaky Test] TestRestartCluster#testClusterRestart > -- > > Key: HBASE-25030 > URL: https://issues.apache.org/jira/browse/HBASE-25030 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[ https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan reassigned HBASE-25025: - Assignee: Reid Chan > [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp > - > > Key: HBASE-25025 > URL: https://issues.apache.org/jira/browse/HBASE-25025 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[ https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195389#comment-17195389 ] Reid Chan commented on HBASE-25025: --- TestFromClientSideWithCoprocessor#testCheckAndDeleteWithCompareOp is related to this one. > [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp > - > > Key: HBASE-25025 > URL: https://issues.apache.org/jira/browse/HBASE-25025 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
