[jira] [Created] (HBASE-20652) Remove internal uses of some deprecated MasterObserver hooks
Appy created HBASE-20652: Summary: Remove internal uses of some deprecated MasterObserver hooks Key: HBASE-20652 URL: https://issues.apache.org/jira/browse/HBASE-20652 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Remove internal uses of these hooks: - preModifyNamespace - postModifyNamespace - preModifyTable - postModifyTable - preModifyTableAction - postCompletedModifyTableAction -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20567) Given both old and new descriptors to pre/post hooks of modify operations for table and namespace
Appy created HBASE-20567:
Summary: Given both old and new descriptors to pre/post hooks of
modify operations for table and namespace
Key: HBASE-20567
URL: https://issues.apache.org/jira/browse/HBASE-20567
Project: HBase
Issue Type: Improvement
Reporter: Appy
In postModify* hooks like {{postModifyX(..., Descriptor newDesc)}}, there's no
way of getting the old descriptor which was there before modification happened.
Having both old and new descriptors will make the hooks more useful.
We felt the need when we wanted to audit certain events but there was no way of
deducing them by just seeing 'after-state' of modification.
To keep the method signatures consistent, i have modified both pre and post
hooks with new arguments which are well named and commented.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-17730) [DOC] Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-17730. -- Resolution: Fixed > [DOC] Migration to 2.0 for coprocessors > > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Assignee: Appy >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-17730.master.001.patch > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20249) Investigate behavior of hbase.client.operation.timeout
Appy created HBASE-20249: Summary: Investigate behavior of hbase.client.operation.timeout Key: HBASE-20249 URL: https://issues.apache.org/jira/browse/HBASE-20249 Project: HBase Issue Type: Bug Reporter: Appy hbase.client.operation.timeout should be hard limit on client operations, and independent of number of retires. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-17449) Add explicit document on different timeout settings
[ https://issues.apache.org/jira/browse/HBASE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-17449. -- Resolution: Fixed > Add explicit document on different timeout settings > --- > > Key: HBASE-17449 > URL: https://issues.apache.org/jira/browse/HBASE-17449 > Project: HBase > Issue Type: Improvement > Components: documentation >Reporter: Yu Li >Assignee: Peter Somogyi >Priority: Critical > Fix For: 2.0.0 > > Attachments: 0001-Standalone.patch, HBASE-17449.master.002.patch, > WIP-HBASE-17499.master.001.patch > > > Currently we have more than one timeout settings, mainly includes: > * hbase.rpc.timeout > * hbase.client.operation.timeout > * hbase.client.scanner.timeout.period > And in latest branch-1 or master branch code, we will have two other > properties: > * hbase.rpc.read.timeout > * hbase.rpc.write.timeout > However, in current refguid we don't have explicit instruction on the > difference of these timeout settings (there're explanations for each > property, but no instruction on when to use which) > In my understanding, for RPC layer timeout, or say each rpc call: > * Scan (openScanner/next): controlled by hbase.client.scanner.timeout.period > * Other operations: >1. For released versions: controlled by hbase.rpc.timeout >2. For 1.4+ versions: read operation controlled by hbase.rpc.read.timeout, > write operation controlled by hbase.rpc.write.timeout, or hbase.rpc.timeout > if the previous two are not set. > And hbase.client.operation.timeout is a higher-level control counting retry > in, or say the overall control for one user call. > After this JIRA, I hope when users ask questions like "What settings I should > use if I don't want to wait for more than 1 second for a single > put/get/scan.next call", we could give a neat answer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20181) Logging and minor logic improvements in BackupLogCleaner
[ https://issues.apache.org/jira/browse/HBASE-20181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-20181. -- Resolution: Fixed > Logging and minor logic improvements in BackupLogCleaner > > > Key: HBASE-20181 > URL: https://issues.apache.org/jira/browse/HBASE-20181 > Project: HBase > Issue Type: Improvement > Components: hbase >Affects Versions: 2.0.0 >Reporter: BELUGA BEHR >Assignee: BELUGA BEHR >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HBASE-20181.1.patch > > > # Simplify code > # Use re-usable libraries > # Use SLF4J parameter logging format > # Clean up some log messages -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20185) Fix ACL check for MasterRpcServices#execProcedure
Appy created HBASE-20185: Summary: Fix ACL check for MasterRpcServices#execProcedure Key: HBASE-20185 URL: https://issues.apache.org/jira/browse/HBASE-20185 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Mailing thread ref: [http://mail-archives.apache.org/mod_mbox/hbase-dev/201803.mbox/%3CCAAjhxrriGy_UXpC4iHCSyBB18iAbjU3Y2%2BnjQ-66i9kPPCrPRQ%40mail.gmail.com%3E] TLDR; HBASE-19400 messed up perms required for flushing a table. Looks like flush and snapshot procedures are already doing permissions check as part of preTableFlush/preSnapshot hooks. However, LogRollMasterProcedureManager is missing access checks ([~elserj], can someone look at it?) With that, it makes no sense to put an ADMIN perm requirement which was added by me in HBASE-19400. Removing it. However, to make things better for future, i have made few design changes which will ensure 1) perm checks don't slip by mistake, 2) a suitable placeholder for checks for flush & snapshot when we remove AccessController for good. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
[ https://issues.apache.org/jira/browse/HBASE-19114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19114. -- Resolution: Fixed > Split out o.a.h.h.zookeeper from hbase-server and hbase-client > -- > > Key: HBASE-19114 > URL: https://issues.apache.org/jira/browse/HBASE-19114 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19114.master.001.patch, > HBASE-19114.master.002.patch, HBASE-19114.master.003.patch, > HBASE-19114.master.004.patch, HBASE-19114.master.005.patch, > HBASE-19114.master.006.patch, HBASE-19114.master.007.patch, > HBASE-19114.master.008.patch, HBASE-19114_addendum.patch > > > Changes so far: > - Moved DrainingServerTracker and RegionServerTracker to > hbase-server:o.a.h.h.master. > - Moved SplitOrMergeTracker to oahh.master (because it depends on a PB) > - Moving hbase-client:oahh.zookeeper.* to hbase-zookeeper module. After > [~Apache9]'s cleanup (HBASE-19200), hbase-client doesn't need them anymore > (except 3 classes). > - Renamed some classes to use a consistent naming for classes - ZK instead of > mix of ZK, Zk , ZooKeeper. Couldn't rename following public classes: > MiniZooKeeperCluster, ZooKeeperConnectionException. Left RecoverableZooKeeper > for lack of better name. (suggestions?) > - Sadly, can't move tests out because they depend on HBaseTestingUtility > (which defeats part of the purpose - trimming down hbase-server tests. We > need to promote more use of mocks in our tests) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19400) Add missing security checks in MasterRpcServices
[ https://issues.apache.org/jira/browse/HBASE-19400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reopened HBASE-19400: -- Uploading patch for branch-1 Patch is both for this jira and few lines in HBASE-19401 > Add missing security checks in MasterRpcServices > > > Key: HBASE-19400 > URL: https://issues.apache.org/jira/browse/HBASE-19400 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-beta-1 >Reporter: Balazs Meszaros >Assignee: Appy >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19400.branch-1.001.patch, > HBASE-19400.master.001.patch, HBASE-19400.master.002.patch, > HBASE-19400.master.003.patch, HBASE-19400.master.004.patch, > HBASE-19400.master.004.patch, HBASE-19400.master.005.patch, > HBASE-19400.master.006.patch, HBASE-19400.master.007.patch, > HBASE-19400.master.007.patch > > > The following RPC methods in MasterRpcServices do not have ACL check for > ADMIN rights. > - normalize > - setNormalizerRunning > - runCatalogScan > - enableCatalogJanitor > - runCleanerChore > - setCleanerChoreRunning > - execMasterService > - execProcedure > - execProcedureWithRet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20010) Rename Master#checkInitialized to Master#isAcceptingRpc; Ensure called for all RPC methods
Appy created HBASE-20010: Summary: Rename Master#checkInitialized to Master#isAcceptingRpc; Ensure called for all RPC methods Key: HBASE-20010 URL: https://issues.apache.org/jira/browse/HBASE-20010 Project: HBase Issue Type: Bug Reporter: Appy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19993) Publish tests jar for hbase-zookeeper in bin tarball
Appy created HBASE-19993:
Summary: Publish tests jar for hbase-zookeeper in bin tarball
Key: HBASE-19993
URL: https://issues.apache.org/jira/browse/HBASE-19993
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
Since {{HBTU extends HBZKTU}} (such short forms! i know!), we need to publish
hbase-zookeeper's tests jar too. Many IT tests use HBTU.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19970) Remove unused functions from TableAuthManager
Appy created HBASE-19970: Summary: Remove unused functions from TableAuthManager Key: HBASE-19970 URL: https://issues.apache.org/jira/browse/HBASE-19970 Project: HBase Issue Type: Task Reporter: Appy Assignee: Appy Functions deleted in TableAuthManager: - setTableUserPermissions - setTableGroupPermissions - setNamespaceUserPermissions - setNamespaceGroupPermissions - writeTableToZooKeeper - writeNamespaceToZooKeeper To make sure it was not a bug, and that relevant functionality moved to some alternate code path, tried to find out why and when these functions went out of use. But just couldn't figure out...until i reached the patch which added them. Looks like they were dead functions to start with :) Jira which added them: HBASE-8409. Commit id: ac10b3c13d6b66e12d0c9601204b01dfa525ed19 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19915) From split/ merge procedures daughter/ merged regions get created in OFFLINE state
[ https://issues.apache.org/jira/browse/HBASE-19915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19915. -- Resolution: Fixed > From split/ merge procedures daughter/ merged regions get created in OFFLINE > state > -- > > Key: HBASE-19915 > URL: https://issues.apache.org/jira/browse/HBASE-19915 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-beta-1 >Reporter: Umesh Agashe >Assignee: Umesh Agashe >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: hbase-19915.addendum.patch, > hbase-19915.master.001.patch, hbase-19915.master.001.patch > > > See HBASE-19530. When regions are created initial state should be CLOSED. Bug > was discovered while debugging flaky test > TestSplitTableRegionProcedure#testRollbackAndDoubleExecution with numOfSteps > set to 4. After updating daughter regions in meta when master is restarted, > startup sequence of master assigns all OFFLINE regions. As daughter regions > are stored with OFFLINE state, daughter regions are assigned. This is > followed by re-assignment of daughter regions from resumed > SplitTableRegionProcedure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19866) TestRegionServerReportForDuty doesn't timeout
Appy created HBASE-19866: Summary: TestRegionServerReportForDuty doesn't timeout Key: HBASE-19866 URL: https://issues.apache.org/jira/browse/HBASE-19866 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19862) Fix TestTokenAuthentication - fake RegionCoprocessorEnvironment is not of type HasRegionServerServices
Appy created HBASE-19862:
Summary: Fix TestTokenAuthentication - fake
RegionCoprocessorEnvironment is not of type HasRegionServerServices
Key: HBASE-19862
URL: https://issues.apache.org/jira/browse/HBASE-19862
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
We have temporary HasRegionServerServices (added in HBASE-19007) and concept of
CoreCoprocessors which require that whichever *CoprocessorEnvironment they get,
it should also implement HasRegionServerServices.
This test builds mock RegionCpEnv for TokenProvider (RegionCoprocessor), but it
falls short of what's expected and results in following exceptions in test logs
{noformat}
2018-01-25 14:38:54,855 ERROR [TokenServer:d9a9782cd075,39492,1516891133911]
helpers.MarkerIgnoringBase(159): Aborting on:
org.apache.hadoop.hbase.security.token.TestTokenAuthentication$TokenServer$2
cannot be cast to org.apache.hadoop.hbase.coprocessor.HasRegionServerServices
java.lang.ClassCastException:
org.apache.hadoop.hbase.security.token.TestTokenAuthentication$TokenServer$2
cannot be cast to org.apache.hadoop.hbase.coprocessor.HasRegionServerServices
at
org.apache.hadoop.hbase.security.token.TokenProvider.start(TokenProvider.java:70)
at
org.apache.hadoop.hbase.security.token.TestTokenAuthentication$TokenServer.initialize(TestTokenAuthentication.java:275)
at
org.apache.hadoop.hbase.security.token.TestTokenAuthentication$TokenServer.run(TestTokenAuthentication.java:347)
{noformat}
Patch adds the missing interface to the mock. Also, uses Mockito to mock the
interfaces rather the crude way.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19860) Run branch-2 nightly job more frequently
[ https://issues.apache.org/jira/browse/HBASE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19860. -- Resolution: Fixed > Run branch-2 nightly job more frequently > > > Key: HBASE-19860 > URL: https://issues.apache.org/jira/browse/HBASE-19860 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-19860.branch-2.001.patch > > > For time being, running the job more frequently will give us more data faster > to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19860) Run branch-2 nightly job more frequently
Appy created HBASE-19860: Summary: Run branch-2 nightly job more frequently Key: HBASE-19860 URL: https://issues.apache.org/jira/browse/HBASE-19860 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy For time being, running the job more frequently will give us more data faster to debug flaky tests . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19823) Make RawCellBuilderFactory LimitedPrivate.UNITTEST
Appy created HBASE-19823: Summary: Make RawCellBuilderFactory LimitedPrivate.UNITTEST Key: HBASE-19823 URL: https://issues.apache.org/jira/browse/HBASE-19823 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19820) Restore public API compat of MiniHBaseCluster
Appy created HBASE-19820: Summary: Restore public API compat of MiniHBaseCluster Key: HBASE-19820 URL: https://issues.apache.org/jira/browse/HBASE-19820 Project: HBase Issue Type: Improvement Reporter: Appy HBASE-18352 removed a public constructor of MiniHBaseCluster. Adding it back. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19819) Decide the place of Checkstyle in build flow
Appy created HBASE-19819:
Summary: Decide the place of Checkstyle in build flow
Key: HBASE-19819
URL: https://issues.apache.org/jira/browse/HBASE-19819
Project: HBase
Issue Type: Improvement
Reporter: Appy
Ref: https://issues.apache.org/jira/browse/HBASE-19780
Main questions:
# Should checkstyle (CS) be part of {{mvn install}}. On master, mvn install
(without clean) takes ~3 min and {{mvn checkstyle:checkstyle}} takes ~2min.
I think the reason it's not part of default build might be - our project isn't
clean, and failing because of existing 10k CS issues is useless. Maybe there's
no trivial way of reporting just the new CS issues, and that's why we depend on
QA (which gives just the diff) for checkstyle?
# How to avoid regressions in modules which have been sanitized by [~Jan
Hentschel]. Here's a suggestion building on his:
** Let's add a recommendation in documentation that run mvn checkstyle:check
before submitting patches since it'll catch CS violations in modules which are
perfectly clean.
** Add checkstyle:check as part of main pre-commit build. If there is any
violation in these clean modules (towards which you have put great effort),
then the pre-commit will fail also for the mvn install step, which is an
important one. Thus, clean CK in these modules become hard pre-commit
requirement indirectly.
Let's put a note on dev@ proposing these changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19789) Not exclude flaky tests from nightly builds
[ https://issues.apache.org/jira/browse/HBASE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19789. -- Resolution: Fixed Fix Version/s: 2.0.0-beta-2 1.2.7 1.5.0 1.4.1 1.3.2 > Not exclude flaky tests from nightly builds > --- > > Key: HBASE-19789 > URL: https://issues.apache.org/jira/browse/HBASE-19789 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Major > Fix For: 1.3.2, 1.4.1, 1.5.0, 1.2.7, 2.0.0-beta-2 > > Attachments: HBASE-19789.master.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19789) Exclude flaky tests from nightly branch-2 runs
Appy created HBASE-19789: Summary: Exclude flaky tests from nightly branch-2 runs Key: HBASE-19789 URL: https://issues.apache.org/jira/browse/HBASE-19789 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19780) Fix build in branch-2
Appy created HBASE-19780:
Summary: Fix build in branch-2
Key: HBASE-19780
URL: https://issues.apache.org/jira/browse/HBASE-19780
Project: HBase
Issue Type: Bug
Reporter: Appy
Not able to run following command successfully:
{{mvn -DskipTests install site
-Dmaven.repo.local=/Users/appy/Desktop/temp_repo}}
Use a clean separate repo so that existing packages don't pollute the build.
Error is following.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on project
hbase: failed to get report for org.apache.maven.plugins:maven-javadoc-plugin:
Failed to execute goal
org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle) on
project hbase-error-prone: Execution checkstyle of goal
org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check failed: Plugin
org.apache.maven.plugins:maven-checkstyle-plugin:2.17 or one of its
dependencies could not be resolved: Failure to find
org.apache.hbase:hbase-checkstyle:jar:2.0.0-beta-1 in
http://repository.apache.org/snapshots/ was cached in the local repository,
resolution will not be reattempted until the update interval of
apache.snapshots has elapsed or updates are forced -> [Help 1]
{noformat}
Note that master build goes pass this point.
Need to figure out what's the difference and fix the overall build.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-19773) Adding javadoc around getting instance of RawCellBuilder
[ https://issues.apache.org/jira/browse/HBASE-19773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19773. -- Resolution: Fixed > Adding javadoc around getting instance of RawCellBuilder > > > Key: HBASE-19773 > URL: https://issues.apache.org/jira/browse/HBASE-19773 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-19773.master.001.patch, > HBASE-19773.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19773) RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience
Appy created HBASE-19773: Summary: RawCellBuilder is LP.Coproc, it's factory should have save InterfaceAudience Key: HBASE-19773 URL: https://issues.apache.org/jira/browse/HBASE-19773 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19715) Fix timing out test TestMultiRespectsLimits
Appy created HBASE-19715: Summary: Fix timing out test TestMultiRespectsLimits Key: HBASE-19715 URL: https://issues.apache.org/jira/browse/HBASE-19715 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-10092) Move to slf4j
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-10092. -- Resolution: Fixed > Move to slf4j > - > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Balazs Meszaros >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: 10092.txt, 10092v2.txt, > HBASE-10092-addendum-2.master.002.patch, HBASE-10092-addendum.patch, > HBASE-10092-preview-v0.patch, HBASE-10092.master.001.patch, > HBASE-10092.master.002.patch, HBASE-10092.master.003.patch, > HBASE-10092.master.004.patch, HBASE-10092.patch > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-10092) Move to slf4j
[ https://issues.apache.org/jira/browse/HBASE-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reopened HBASE-10092: -- > Move to slf4j > - > > Key: HBASE-10092 > URL: https://issues.apache.org/jira/browse/HBASE-10092 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: Balazs Meszaros >Priority: Critical > Fix For: 2.0.0-beta-1 > > Attachments: 10092.txt, 10092v2.txt, HBASE-10092-addendum.patch, > HBASE-10092-preview-v0.patch, HBASE-10092.master.001.patch, > HBASE-10092.master.002.patch, HBASE-10092.master.003.patch, > HBASE-10092.master.004.patch, HBASE-10092.patch > > > Allows logging with less friction. See http://logging.apache.org/log4j/2.x/ > This rather radical transition can be done w/ minor change given they have an > adapter for apache's logging, the one we use. They also have and adapter for > slf4j so we likely can remove at least some of the 4 versions of this module > our dependencies make use of. > I made a start in attached patch but am currently stuck in maven dependency > resolve hell courtesy of our slf4j. Fixing will take some concentration and > a good net connection, an item I currently lack. Other TODOs are that will > need to fix our little log level setting jsp page -- will likely have to undo > our use of hadoop's tool here -- and the config system changes a little. > I will return to this project soon. Will bring numbers. > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19589) New regions should always be added with state CLOSED
Appy created HBASE-19589: Summary: New regions should always be added with state CLOSED Key: HBASE-19589 URL: https://issues.apache.org/jira/browse/HBASE-19589 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Followup of HBASE-19530. Looks like it missed a code path. Looked deeper into MetaTableAccessor and usages of some of its functions to correctly set initial state as CLOSED. Other things - Removed unused functions. - Changed logging to parameterized -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19571) Minor refactor of Nightly run scripts
[ https://issues.apache.org/jira/browse/HBASE-19571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19571. -- Resolution: Fixed Fix Version/s: 2.0.0-beta-1 > Minor refactor of Nightly run scripts > - > > Key: HBASE-19571 > URL: https://issues.apache.org/jira/browse/HBASE-19571 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19571.master.001.patch > > > Was trying to split out common code in separate lib, but it's not at all > trivial. The way to do it for declarative syntax Jenkinsfile is by using > Shared Libraries which requires separate repo! > The patch ended up being just naming refactors. > Renames OUTPUTDIR to OUTPUT_DIR > Renames OUTPUT_RELATIVE to OUTPUT_DIR_RELATIVE -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19583) Delete EOL branches - branch-1.0 and branch-1.1
Appy created HBASE-19583: Summary: Delete EOL branches - branch-1.0 and branch-1.1 Key: HBASE-19583 URL: https://issues.apache.org/jira/browse/HBASE-19583 Project: HBase Issue Type: Bug Reporter: Appy Priority: Minor wdys [~enis] [~ndimiduk]? This needs updating too - http://hbase.apache.org/book.html#_release_managers. I think we should mention RMs for all branches, even eols, if only in other table. Good way to recognize past efforts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19571) Minor refactor of Nightly run scripts
Appy created HBASE-19571: Summary: Minor refactor of Nightly run scripts Key: HBASE-19571 URL: https://issues.apache.org/jira/browse/HBASE-19571 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Priority: Minor Was trying to split out common code in separate lib, but it's not at all trivial. The way to do it for declarative syntax Jenkinsfile is by using Shared Libraries which requires separate repo! The patch ended up being just naming refactors. Renames OUTPUTDIR to OUTPUT_DIR Renames OUTPUT_RELATIVE to OUTPUT_DIR_RELATIVE -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19570) Add hadoop3 tests to Nightly master/branch-2 runs
Appy created HBASE-19570: Summary: Add hadoop3 tests to Nightly master/branch-2 runs Key: HBASE-19570 URL: https://issues.apache.org/jira/browse/HBASE-19570 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19530) New regions should always be added with state CLOSED
Appy created HBASE-19530: Summary: New regions should always be added with state CLOSED Key: HBASE-19530 URL: https://issues.apache.org/jira/browse/HBASE-19530 Project: HBase Issue Type: Bug Reporter: Appy We shouldn't add regions with state null. In case of failures and recovery, it's not possible to determine what did it mean and things become uncertain. All operations should add regions in a well defined state. For now, we'll set the default to CLOSED, since whatever ops are adding new regions, they would anyways be enabling them explicitly if needed. fyi: [~stack] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19529) Handle null states in AM
Appy created HBASE-19529: Summary: Handle null states in AM Key: HBASE-19529 URL: https://issues.apache.org/jira/browse/HBASE-19529 Project: HBase Issue Type: Bug Reporter: Appy >From debugging in HBASE-19457, found some questions that need concrete answers: 1) What does a region state of null in meta means? Currently AM treats it as OFFLINE 2) What does a table state of null in meta means? Currently TSM treats it as ENABLED. More importantly, we need to fix holes in AM so that our state machine is well defined and doesn't end up in these uncertainties. Figuring out answers to above questions will help in that direction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19500) Make RSGroupInfo immutable
Appy created HBASE-19500:
Summary: Make RSGroupInfo immutable
Key: HBASE-19500
URL: https://issues.apache.org/jira/browse/HBASE-19500
Project: HBase
Issue Type: Bug
Reporter: Appy
HBASE-19483 added CP hooks to expose RSGroupInfo.
First, we should at least change [hbase-client] RSGroupInfo to immutable +
builder pattern like we have done for so many other things.
What say [~Apache9]
Then, few questions need figuring out:
- Should hooks be allowed to change RSGroupInfo.
Probably not? Then making it immutable would be necessary and sufficient
- Can we remove {{if(((MasterEnvironment)getEnvironment()).supportGroupCPs) }}
in so many places since CP in 2.0 are already broken left and right (and we'll
have to solve legacy issue more holistically) What say [~anoop.hbase]?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19491) Exclude flaky tests from nightly master run
Appy created HBASE-19491:
Summary: Exclude flaky tests from nightly master run
Key: HBASE-19491
URL: https://issues.apache.org/jira/browse/HBASE-19491
Project: HBase
Issue Type: Improvement
Reporter: Appy
Assignee: Appy
I was of the opinion that nightly master runs were excluding flaky tests after
seeing
https://github.com/apache/hbase/blob/856ee283faf003404e8925006ce0e591c4eba600/dev-support/Jenkinsfile#L54
few days ago.
After looking at our new set of scripts again and better understanding them,
looks like that's not enough.
We need to set
{code}DOCKER_EXTRAARGS=--env=EXCLUDE_TESTS_URL=${EXCLUDE_TESTS_URL}{code} like
in https://builds.apache.org/job/PreCommit-HBASE-Build/configure
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19489) Check against only the latest maintenance release in pre-commit hadoopcheck.
Appy created HBASE-19489:
Summary: Check against only the latest maintenance release in
pre-commit hadoopcheck.
Key: HBASE-19489
URL: https://issues.apache.org/jira/browse/HBASE-19489
Project: HBase
Issue Type: Improvement
Reporter: Appy
Assignee: Appy
Priority: Minor
(copied from dev thread)
{color:green}
| +1| hadoopcheck | 52m 1s |Patch does not cause any errors with
Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 2.7.4 or 3.0.0-alpha4. |
{color}
Almost 1 hr to check against 10 versions. And it's only going to increase as
more 2.6.x, 2.7.x and 3.0.x releases come out.
Suggestion here is simple, let's check against only the latest maintenance
release for each minor version i.e. 2.6.5, 2.7.4 and 3.0.0-alpha4.
Advantage: Save ~40 min on pre-commit time.
Justification:
- We only do compile checks. Maintenance releases are not supposed to be doing
API breaking changes. So checking against maintenance release for each minor
version should be enough.
- We rarely see any hadoop check -1, and most recent ones have been due to 3.0.
These will still be caught.
- Nightly can still check against all hadoop versions (since nightlies are
supposed to do holistic testing)
- Analyzing 201 precommits from 10100 (11/29) - 10300 (12/8) (10 days):
138 had +1 hadoopcheck
15 had -1 hadoopcheck
(others probably failed even before that - merge issue, etc)
Spot checking some
failures:(10241,10246,10225,10269,10151,10156,10184,10250,10298,10227,10294,10223,10251,10119,10230)
10241: All 2.6.x failed. Others didn't run
10246: All 10 versions failed.
10184: All 2.6.x and 2.7.x failed. Others didn't run
10223: All 10 versions failed
10230: All 2.6.x failed. Others didn't run
Common pattern being, all maintenance versions fail together.
(idk, why sometimes 2.7.* are not reported if 2.6.* fail, but that's irrelevant
to this discussion).
What do you say - only check latest maintenance releases in precommit (and let
nightlies do holistic testing against all versions)?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19457) Debugging TestTruncateTableProcedure failures
Appy created HBASE-19457: Summary: Debugging TestTruncateTableProcedure failures Key: HBASE-19457 URL: https://issues.apache.org/jira/browse/HBASE-19457 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19454) Debugging TestDistributedLogSplitting#testThreeRSAbort
Appy created HBASE-19454: Summary: Debugging TestDistributedLogSplitting#testThreeRSAbort Key: HBASE-19454 URL: https://issues.apache.org/jira/browse/HBASE-19454 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy It's flaky tests. Attaching a patch to add some general logging to help in future tests' debugging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19436) Remove Jenkinsfile from some branches
[ https://issues.apache.org/jira/browse/HBASE-19436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19436. -- Resolution: Fixed Fix Version/s: HBASE-14070 > Remove Jenkinsfile from some branches > - > > Key: HBASE-19436 > URL: https://issues.apache.org/jira/browse/HBASE-19436 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Fix For: HBASE-14070 > > Attachments: HBASE-19436.HBASE-14070.HLC.001.patch > > > Some of the branches in https://builds.apache.org/job/HBase%20Nightly/ don't > need to be build nightly since they are not in active development. Let's > remove them to reduce clutter and free up restricted resources. Suggested > branches: > HBASE-14070.HLC : It got it as a part of forking from master where Sean's > work was in progress. Wasn't added intentionally. Removing Jenkinsfile from > it. > HBASE-19297 : deleted as part of closing the jira itself. > (todo: others?) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19382) Update report-flakies.py script to handle yetus builds
[ https://issues.apache.org/jira/browse/HBASE-19382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19382. -- Resolution: Fixed > Update report-flakies.py script to handle yetus builds > -- > > Key: HBASE-19382 > URL: https://issues.apache.org/jira/browse/HBASE-19382 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19382.master.001.patch > > > With move to new nightly build which uses yetus > (https://builds.apache.org/job/HBase%20Nightly/job/master/), current > report-flakies.py is not able to build test report since maven output is not > in consoleText anymore. Update script to accept both traditional builds > (maven output in consoleTest, for flakies runner job) and yetus builds > (maven output in artifacts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19382) Update report-flakies.py script to handle yetus builds
Appy created HBASE-19382: Summary: Update report-flakies.py script to handle yetus builds Key: HBASE-19382 URL: https://issues.apache.org/jira/browse/HBASE-19382 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy With move to new nightly build which uses yetus (https://builds.apache.org/job/HBase%20Nightly/job/master/), current report-flakies.py is not able to build test report since maven output is not in consoleText anymore. Update script to accept both traditional builds (maven output in consoleTest, for flakies runner job) and yetus builds (maven output in artifacts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19367) Refactoring in RegionStates, and RSProcedureDispatcher
Appy created HBASE-19367: Summary: Refactoring in RegionStates, and RSProcedureDispatcher Key: HBASE-19367 URL: https://issues.apache.org/jira/browse/HBASE-19367 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Priority: Minor Working on a bug fix, was in these parts for first time to understand new AM and trying to make sense of things. Did a few improvements on the way. - Adding javadoc comments - Bug: ServerStateNode#regions is HashSet but there's no synchronization to prevent concurrent add/removes. Let's use concurrent set instead. - Use getRegionsInTransitionCount() directly to avoid instead of getRegionsInTransition().size() because the latter copies everything into a new array - what a waste for just the size. - There's mixed use of getRegionNode and getRegionStateNode for same return type - RegionStateNode. Changing everything to getRegionStateNode. Similarly rename other *RegionNode() fns to *RegionStateNode(). - RegionStateNode#transitionState() return value is useless since it always returns it's first param. - Other minor improvements -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19335) Fix waitUntilAllRegionsAssigned
Appy created HBASE-19335:
Summary: Fix waitUntilAllRegionsAssigned
Key: HBASE-19335
URL: https://issues.apache.org/jira/browse/HBASE-19335
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
Found when debugging flaky test TestRegionObserverInterface#testRecovery.
In the end, the test does the following:
- Kills the RS
- Waits for all regions to be assigned
- Some validation (unrelated)
- Cleanup: delete table.
{noformat}
cluster.killRegionServer(rs1.getRegionServer().getServerName());
Threads.sleep(1000); // Let the kill soak in.
util.waitUntilAllRegionsAssigned(tableName);
LOG.info("All regions assigned");
verifyMethodResult(SimpleRegionObserver.class,
new String[] { "getCtPreReplayWALs", "getCtPostReplayWALs",
"getCtPreWALRestore",
"getCtPostWALRestore", "getCtPrePut", "getCtPostPut" },
tableName, new Integer[] { 1, 1, 2, 2, 0, 0 });
} finally {
util.deleteTable(tableName);
table.close();
}
}
{noformat}
However, looking at test logs, found that we had overlapping Assigns with
Unassigns. As a result, regions ended up 'stuck in RIT' and the test timeout.
Assigns were from the ServerCrashRecovery and Unassigns were from the
deleteTable cleanup.
Which begs the question, why did HBTU.waitUntilAllRegionsAssigned(tableName)
not wait until recovery was complete.
Answer: Looks like that function is only meant for sunny scenarios but not for
crashes. It iterates over meta and just [checks for *some value* in the server
column|https://github.com/apache/hbase/blob/cdc2bb17ff38dcbd273cf501aea565006e995a06/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java#L3421]
which is obviously present and equal to the server that was just killed.
This bug must be affecting other fault tolerance tests too and fixing it may
fix more than just one test, hopefully.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19319) Fix bug in synchronizing over ProcedureEvent
Appy created HBASE-19319:
Summary: Fix bug in synchronizing over ProcedureEvent
Key: HBASE-19319
URL: https://issues.apache.org/jira/browse/HBASE-19319
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
Following synchronizes over local variable rather than the original
ProcedureEvent object. Clearly a bug since this code block won't follow
exclusion with many of the synchronized methods in ProcedureEvent class.
{code}
@Override
public void wakeEvents(final int count, final ProcedureEvent... events) {
final boolean traceEnabled = LOG.isTraceEnabled();
schedLock();
try {
int waitingCount = 0;
for (int i = 0; i < count; ++i) {
final ProcedureEvent event = events[i];
synchronized (event) {
if (!event.isReady()) {
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19291) Use common header and footer for JSP pages
Appy created HBASE-19291: Summary: Use common header and footer for JSP pages Key: HBASE-19291 URL: https://issues.apache.org/jira/browse/HBASE-19291 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Use header and footer in our *.jsp pages to avoid unnecessary redundancy (copy-paste of code) Misc edits: - Due to redundancy, new additions make it to some places but not others. For eg there are missing links to "/logLevel", "/processRS.jsp" in few places. - Fix processMaster.jsp wrongly pointing to rs-status instead of master-status (probably due to copy paste from processRS.jsp) - Deleted a bunch of extraneous "" in processMaster.jsp & processRS.jsp - Added missing tag in snapshot.jsp - Deleted fossils of html5shiv.js. It's uses and the js itself were deleted in the commit "819aed4ccd073d818bfef5931ec8d248bfae5f1f" - Fixed wrongly matched heading tags - Deleted some unused variables Tested: Ran standalone cluster and opened each page to make sure it looked right. Sidenote: Looks like HBASE-3835 started the work of converting from jsp to jamon, but the work didn't finish. Now we have a mix of jsp and jamon. Needs reconciling, but later. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19246) Trivial fix in findHangingTests.py
[ https://issues.apache.org/jira/browse/HBASE-19246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19246. -- Resolution: Fixed Fix Version/s: 2.0.0-beta-1 > Trivial fix in findHangingTests.py > -- > > Key: HBASE-19246 > URL: https://issues.apache.org/jira/browse/HBASE-19246 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy >Priority: Trivial > Fix For: 2.0.0-beta-1 > > Attachments: patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19246) Trivial fix in findHangingTests.py
Appy created HBASE-19246: Summary: Trivial fix in findHangingTests.py Key: HBASE-19246 URL: https://issues.apache.org/jira/browse/HBASE-19246 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Priority: Trivial -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19089. -- Resolution: Fixed > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Appy >Assignee: Appy > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-19089.master.001.patch, > HBASE-19089.master.002.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-17591) Update documentation to say distributed log replay defaults to 'false'
[
https://issues.apache.org/jira/browse/HBASE-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy resolved HBASE-17591.
--
Resolution: Invalid
Assignee: Appy
> Update documentation to say distributed log replay defaults to 'false'
> --
>
> Key: HBASE-17591
> URL: https://issues.apache.org/jira/browse/HBASE-17591
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Aki Ariga
>Assignee: Appy
>Priority: Trivial
> Fix For: 2.0.0
>
>
> As consequence of HBASE-12577, `hbase.master.distributed.log.replay` is no
> longer default true. But in the documentation, it is still noted as default
> true as follows:
> {quote}
> To enable distributed log replay, set hbase.master.distributed.log.replay to
> true. This will be the default for HBase 0.99 (HBASE-10888).
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19124) Move HBase-Nightly test from JenkinsFile to a script in dev-support
Appy created HBASE-19124: Summary: Move HBase-Nightly test from JenkinsFile to a script in dev-support Key: HBASE-19124 URL: https://issues.apache.org/jira/browse/HBASE-19124 Project: HBase Issue Type: Bug Reporter: Appy ref: https://issues.apache.org/jira/browse/HBASE-19089?focusedCommentId=16221551=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16221551 Moving that jenkins test to script will make it easier to test changes to hbase-assembly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19114) Split out o.a.h.h.zookeeper from hbase-server and hbase-client
Appy created HBASE-19114: Summary: Split out o.a.h.h.zookeeper from hbase-server and hbase-client Key: HBASE-19114 URL: https://issues.apache.org/jira/browse/HBASE-19114 Project: HBase Issue Type: Sub-task Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19101) yetus precommit is not find hanging tests
Appy created HBASE-19101: Summary: yetus precommit is not find hanging tests Key: HBASE-19101 URL: https://issues.apache.org/jira/browse/HBASE-19101 Project: HBase Issue Type: Bug Reporter: Appy Examples of failed QA runs with "-1 unit" but no list of tests: - https://issues.apache.org/jira/browse/HBASE-18925?focusedCommentId=16218515=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16218515 - https://issues.apache.org/jira/browse/HBASE-18905?focusedCommentId=16216763=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16216763 Can use findHangingTests.py to find hanging tests and report them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
Appy created HBASE-19089: Summary: Fix the list of included moduleSets in src and binary tars Key: HBASE-19089 URL: https://issues.apache.org/jira/browse/HBASE-19089 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. The core issue is duplication of list. Let me try to get rid of it by using shaded list of includes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19073) Cleanup CoordinatedStateManager
Appy created HBASE-19073: Summary: Cleanup CoordinatedStateManager Key: HBASE-19073 URL: https://issues.apache.org/jira/browse/HBASE-19073 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Discussion thread on dev@ mailing list. http://mail-archives.apache.org/mod_mbox/hbase-dev/201710.mbox/%3CCAAjhxrqjOg90Fdi73kZZe_Gxtrqq8ff%2B%3DAj_epptO_XO812Abg%40mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19053) Split out o.a.h.h.http from hbase-server into a separate module
Appy created HBASE-19053: Summary: Split out o.a.h.h.http from hbase-server into a separate module Key: HBASE-19053 URL: https://issues.apache.org/jira/browse/HBASE-19053 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19032) Set Content-Type header for patches uploaded by submit-patch.py
[ https://issues.apache.org/jira/browse/HBASE-19032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-19032. -- Resolution: Fixed > Set Content-Type header for patches uploaded by submit-patch.py > --- > > Key: HBASE-19032 > URL: https://issues.apache.org/jira/browse/HBASE-19032 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19032.master.001.patch > > > The patches uploaded by submit-patch.py script get automatically downloaded > when clicked. > Quite annoying. Sometimes it's easier to see the patch in the browser itself. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19032) Set Content-Type header for patches uploaded by submit-patch.py
Appy created HBASE-19032: Summary: Set Content-Type header for patches uploaded by submit-patch.py Key: HBASE-19032 URL: https://issues.apache.org/jira/browse/HBASE-19032 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy The patches uploaded by submit-patch.py script get automatically downloaded when clicked. Quite annoying. Sometimes it's easier to see the patch in the browser itself. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19022) Untangle and split hbase-server module
Appy created HBASE-19022: Summary: Untangle and split hbase-server module Key: HBASE-19022 URL: https://issues.apache.org/jira/browse/HBASE-19022 Project: HBase Issue Type: Improvement Reporter: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18954) Make *CoprocessorHost classes private
Appy created HBASE-18954: Summary: Make *CoprocessorHost classes private Key: HBASE-18954 URL: https://issues.apache.org/jira/browse/HBASE-18954 Project: HBase Issue Type: Sub-task Reporter: Appy Move out configuration name constants (into Coprocessor class?) and made Host classes private. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18944) FIx bad patterns in TestAccessController
Appy created HBASE-18944: Summary: FIx bad patterns in TestAccessController Key: HBASE-18944 URL: https://issues.apache.org/jira/browse/HBASE-18944 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy 1. Looking into the pattern used in TestAccessController & TestWithDisabledAuthorization, i think we can improve it by invoking hooks via CpHosts instead of digging into CP framework's internals. 2. There is something very wrong with TestAccessController. Individual tests are not completely isolated. We have TEST_TABLE which was supposed to be a "constant" for use in basic tests and had a fixed set of perms in setupBeforeClass. However, now some these are changing perms on it. Which has a big possibility of testing bugs. I won't be surprised if things start breaking when we untangle this. Let me file separate jira for that ref: https://issues.apache.org/jira/browse/HBASE-18931?focusedCommentId=16192303=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16192303 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18931) Make ObserverContext an interface and remove private/testing methods
Appy created HBASE-18931: Summary: Make ObserverContext an interface and remove private/testing methods Key: HBASE-18931 URL: https://issues.apache.org/jira/browse/HBASE-18931 Project: HBase Issue Type: Sub-task Reporter: Appy ObserverContext is IA.LimitedPrivate because CP implementations might want to use getEnvironment(), bypass(), complete(), getCaller(), etc. However, they are not supposed to be using the following: - Constructor : Only CPHost can create contexts. - prepare(E env) : this is for internal use only. - createAndPrepare() : ObserverContext is created in context of a CPHost. Third party have use accessing constructor or these factory methods. Internally, these fns are used only by our tests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18926) Cleanup Optional from method params
Appy created HBASE-18926: Summary: Cleanup Optional from method params Key: HBASE-18926 URL: https://issues.apache.org/jira/browse/HBASE-18926 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Duo Zhang ref: Discussion on HBASE-18878 tldr: Optional is not supposed to be used in method params or class fields (http://dolszewski.com/java/java-8-optional-use-cases/). We are doing that in some places. Let's revert that. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18925) Need updated mockito for using java optional
Appy created HBASE-18925: Summary: Need updated mockito for using java optional Key: HBASE-18925 URL: https://issues.apache.org/jira/browse/HBASE-18925 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Came up when i was trying to test HBASE-18878. It kept failing because mock of RpcCall returned null where return type was Optional. Instead, we want it to return Optional.empty(). New mockito versions support this (and other java8 things) - https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2 We use mockito-all which was last released in Dec2014. However, mockito-core has had more than 50 releases after that (https://mvnrepository.com/artifact/org.mockito/mockito-core). We need to change our deps from mockito-all to mockito-core. However that comes with fair breakages, so this is not a simple task of changing pom files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18903) Investigate PreCommit miss on Hadoop-3 compatibility
[ https://issues.apache.org/jira/browse/HBASE-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18903. -- Resolution: Duplicate > Investigate PreCommit miss on Hadoop-3 compatibility > > > Key: HBASE-18903 > URL: https://issues.apache.org/jira/browse/HBASE-18903 > Project: HBase > Issue Type: Bug >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Blocker > Fix For: 3.0.0 > > > We accidentally committed HBASE-18843 without realizing that it broke > compilation against Hadoop 3. The PreCommit check reported that it was OK > against Hadoop3, so we were none-the-wiser. > Appy noticed that the reason for this was that some Hadoop 2.7.1. jars were > actually being used instead of the Hadoop 3 jars. > Need to investigate what the heck is happening to prevent further bad commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy reopened HBASE-18843: -- > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18843-v1.patch, HBASE-18843-v2.patch, > HBASE-18843-v4.patch, HBASE-18843-v5.patch > > > Currently, we copy bulk loaded files to backup one-by-one on a client side > (where backup create runs). This has to be replaced with DistCp copying. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18884) Coprocessor Design Improvements 2 (Follow up of HBASE-17732)
Appy created HBASE-18884:
Summary: Coprocessor Design Improvements 2 (Follow up of
HBASE-17732)
Key: HBASE-18884
URL: https://issues.apache.org/jira/browse/HBASE-18884
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
Creating new jira to track suggestions that came in review
(https://reviews.apache.org/r/62141/) but are not blocker and can be done
separately.
Suggestions by [~apurtell]
- Change {{Service Coprocessor#getService()}} to {{List
Coprocessor#getServices()}}
- I think we overstepped by offering [table resource management via this
interface|https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/CoprocessorEnvironment.java#L57].
There are a lot of other internal resource types which could/should be managed
this way but they are all left up to the implementor. Perhaps we should remove
the table ref management and leave it up to them as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18776) Update coprocessor docs after HBASE-17732
Appy created HBASE-18776: Summary: Update coprocessor docs after HBASE-17732 Key: HBASE-18776 URL: https://issues.apache.org/jira/browse/HBASE-18776 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18696) Fix the problems when introducing the new hbase-mapreduce module
[ https://issues.apache.org/jira/browse/HBASE-18696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18696. -- Resolution: Fixed > Fix the problems when introducing the new hbase-mapreduce module > > > Key: HBASE-18696 > URL: https://issues.apache.org/jira/browse/HBASE-18696 > Project: HBase > Issue Type: Umbrella > Components: mapreduce >Affects Versions: 3.0.0, 2.0.0-alpha-2 >Reporter: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-3 > > > See the comment in HBASE-18640 > https://issues.apache.org/jira/browse/HBASE-18640?focusedCommentId=16143234=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16143234 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18760) Make hbase-shaded-check-invariants part of precommit
Appy created HBASE-18760: Summary: Make hbase-shaded-check-invariants part of precommit Key: HBASE-18760 URL: https://issues.apache.org/jira/browse/HBASE-18760 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Sean Busbey -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18759) Fix hbase-shaded-check-invariants failure
Appy created HBASE-18759:
Summary: Fix hbase-shaded-check-invariants failure
Key: HBASE-18759
URL: https://issues.apache.org/jira/browse/HBASE-18759
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Sean Busbey
Here's the error.
{noformat}
[ERROR] Found artifact with unexpected contents:
'/Users/appy/apache/hbase/hbase-shaded/hbase-shaded-server/target/hbase-shaded-server-3.0.0-SNAPSHOT.jar'
Please check the following and either correct the build or update
the allowed list with reasoning.
about.html
org/eclipse/
org/eclipse/jetty/
org/eclipse/jetty/http/
org/eclipse/jetty/http/encoding.properties
org/eclipse/jetty/http/GZIPContentDecoder$State.class
org/eclipse/jetty/http/HttpFields$Itr.class
org/eclipse/jetty/http/HttpParser$HttpHandler.class
org/eclipse/jetty/http/HttpScheme.class
org/eclipse/jetty/http/HttpURI.class
org/eclipse/jetty/http/pathmap/
org/eclipse/jetty/http/pathmap/PathMappings$1.class
org/eclipse/jetty/http/pathmap/PathSpecGroup.class
org/eclipse/jetty/http/pathmap/PathSpecSet$1.class
org/eclipse/jetty/http/pathmap/RegexPathSpec.class
org/eclipse/jetty/http/PathMap$MappedEntry.class
org/eclipse/jetty/http/QuotedCSV$State.class
org/eclipse/jetty/http/QuotedQualityCSV$1.class
org/eclipse/jetty/http/DateParser.class
org/eclipse/jetty/http/Http1FieldPreEncoder.class
org/eclipse/jetty/http/HttpContent$ContentFactory.class
org/eclipse/jetty/http/HttpFields$1.class
org/eclipse/jetty/http/HttpFields.class
org/eclipse/jetty/http/HttpStatus$Code.class
org/eclipse/jetty/http/pathmap/UriTemplatePathSpec.class
org/eclipse/jetty/http/ResourceHttpContent.class
org/eclipse/jetty/http/BadMessageException.class
org/eclipse/jetty/http/GZIPContentDecoder$1.class
org/eclipse/jetty/http/HttpGenerator$Result.class
org/eclipse/jetty/http/HttpParser$State.class
org/eclipse/jetty/http/HttpURI$State.class
org/eclipse/jetty/http/QuotedCSV.class
org/eclipse/jetty/http/CompressedContentFormat.class
org/eclipse/jetty/http/HttpField$LongValueHttpField.class
org/eclipse/jetty/http/HttpGenerator$2.class
org/eclipse/jetty/http/HttpMethod.class
org/eclipse/jetty/http/HttpParser$ComplianceHandler.class
org/eclipse/jetty/http/HttpParser$RequestHandler.class
org/eclipse/jetty/http/HttpStatus.class
org/eclipse/jetty/http/MetaData$Response.class
org/eclipse/jetty/http/MimeTypes$Type.class
org/eclipse/jetty/http/pathmap/PathMappings.class
org/eclipse/jetty/http/PathMap$PathSet.class
org/eclipse/jetty/http/QuotedQualityCSV.class
org/eclipse/jetty/http/CookieCompliance.class
org/eclipse/jetty/http/GZIPContentDecoder.class
org/eclipse/jetty/http/HttpFields$2.class
org/eclipse/jetty/http/HttpGenerator$State.class
org/eclipse/jetty/http/HttpHeader.class
org/eclipse/jetty/http/HttpParser$IllegalCharacterException.class
org/eclipse/jetty/http/HttpTokens$EndOfContent.class
org/eclipse/jetty/http/HttpURI$1.class
org/eclipse/jetty/http/mime.properties
org/eclipse/jetty/http/PreEncodedHttpField.class
org/eclipse/jetty/http/DateGenerator$1.class
org/eclipse/jetty/http/HostPortHttpField.class
org/eclipse/jetty/http/HttpCompliance.class
org/eclipse/jetty/http/HttpFieldPreEncoder.class
org/eclipse/jetty/http/HttpGenerator$PreparedResponse.class
org/eclipse/jetty/http/HttpParser$1.class
org/eclipse/jetty/http/HttpParser$ResponseHandler.class
org/eclipse/jetty/http/HttpTokens.class
org/eclipse/jetty/http/HttpVersion.class
org/eclipse/jetty/http/MetaData.class
org/eclipse/jetty/http/pathmap/MappedResource.class
org/eclipse/jetty/http/pathmap/PathSpec.class
org/eclipse/jetty/http/pathmap/PathSpecSet.class
org/eclipse/jetty/http/PreEncodedHttpField$1.class
org/eclipse/jetty/http/QuotedCSV$1.class
org/eclipse/jetty/http/Syntax.class
org/eclipse/jetty/http/DateGenerator.class
org/eclipse/jetty/http/HttpContent.class
org/eclipse/jetty/http/HttpField$IntValueHttpField.class
org/eclipse/jetty/http/HttpField.class
org/eclipse/jetty/http/HttpGenerator.class
org/eclipse/jetty/http/MetaData$Request.class
org/eclipse/jetty/http/MimeTypes.class
org/eclipse/jetty/http/pathmap/ServletPathSpec$1.class
org/eclipse/jetty/http/PathMap.class
org/eclipse/jetty/http/DateParser$1.class
org/eclipse/jetty/http/HttpCookie.class
org/eclipse/jetty/http/HttpGenerator$1.class
org/eclipse/jetty/http/HttpHeaderValue.class
org/eclipse/jetty/http/HttpParser$CharState.class
org/eclipse/jetty/http/HttpParser$FieldState.class
org/eclipse/jetty/http/HttpParser.class
org/eclipse/jetty/http/pathmap/ServletPathSpec.class
org/eclipse/jetty/http/PrecompressedHttpContent.class
[jira] [Resolved] (HBASE-14910) Create base hbase-mapred and hbase-io modules, move HTableInputFormat and HTableOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-14910. -- Resolution: Duplicate Fix Version/s: HBASE-18640 > Create base hbase-mapred and hbase-io modules, move HTableInputFormat and > HTableOutputFormat > > > Key: HBASE-14910 > URL: https://issues.apache.org/jira/browse/HBASE-14910 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Nate Edel > Fix For: HBASE-18640 > > Attachments: HBASE-14910.unready.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18688) Upgrade commons-codec to 1.10
Appy created HBASE-18688: Summary: Upgrade commons-codec to 1.10 Key: HBASE-18688 URL: https://issues.apache.org/jira/browse/HBASE-18688 Project: HBase Issue Type: Task Reporter: Appy Assignee: Appy This lib has no new releases in last 3 years. Let's update to latest version 1.10. 1.10 (target) was released on 11/5/14 1.9 (current) was released on 12/20/13 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18509) [HLC] Finishing cleanups
[
https://issues.apache.org/jira/browse/HBASE-18509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy resolved HBASE-18509.
--
Resolution: Fixed
> [HLC] Finishing cleanups
>
>
> Key: HBASE-18509
> URL: https://issues.apache.org/jira/browse/HBASE-18509
> Project: HBase
> Issue Type: Sub-task
>Reporter: Appy
>Assignee: Amit Patel
> Attachments: HBASE-18509.HBASE-14070.HLC.001.patch,
> HBASE-18509.HBASE-14070.HLC.002.patch, HBASE-18509.HBASE-14070.HLC.003.patch,
> HBASE-18509.HBASE-14070.HLC.004.patch, HBASE-18509.HBASE-14070.HLC.005.patch
>
>
> Track all types of cleanups here:
> - (done in 001.patch) --Rename classes to more consistent naming:
> SystemClock, SystemMonotonicClock, HybridLogicalClock--
> - (done in 001.patch) --Move implementations out from Clock interface. It's a
> simple interface of 6 fns but very overloaded right now with everything put
> inside it.--
> - (done in 004.patch) --Maybe encapsulate all clocks in RS/Master into a new
> class. Then RSServices can just have getClocks() function.--
> {noformat}
> class Clocks {
> // all 3 types of clocks.
> // Fns:
> - update(ClockType, timestamp)
> - updateAll(List\)
> - int64 now(ClockType)
> - List\ nowAll()
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18661) [HLC] Move clocks to a separate hbase-clocks module
Appy created HBASE-18661: Summary: [HLC] Move clocks to a separate hbase-clocks module Key: HBASE-18661 URL: https://issues.apache.org/jira/browse/HBASE-18661 Project: HBase Issue Type: Sub-task Reporter: Appy Assignee: Appy I think we can move all clocks related code out of h-common and h-server into a new package which only needs to depend on h-shaded-protocol. If it's possible, I think it'll be much cleaner code layout and dependency structure. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy reopened HBASE-14498:
--
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.2, 1.4.1, 1.5.0, 1.2.7
>
> Attachments: HBASE-14498-addendum.patch,
> HBASE-14498-branch-1.2.patch, HBASE-14498-branch-1.3.patch,
> HBASE-14498-branch-1.3-V2.patch, HBASE-14498-branch-1.4.patch,
> HBASE-14498-branch-1.patch, HBASE-14498.master.001.patch,
> HBASE-14498.master.002.patch, HBASE-14498.patch, HBASE-14498-V2.patch,
> HBASE-14498-V3.patch, HBASE-14498-V4.patch, HBASE-14498-V5.patch,
> HBASE-14498-V6.patch, HBASE-14498-V6.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here
> > region server retrieve master location from zookeeper only when they
> > couldn't connect to Master
[jira] [Created] (HBASE-18640) Move mapreduce out of hbase-server into separate hbase-mapreduce moduel
Appy created HBASE-18640: Summary: Move mapreduce out of hbase-server into separate hbase-mapreduce moduel Key: HBASE-18640 URL: https://issues.apache.org/jira/browse/HBASE-18640 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18457) [HLC] Follow up work of updating clocks (from HBASE-18395)
[ https://issues.apache.org/jira/browse/HBASE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18457. -- Resolution: Duplicate > [HLC] Follow up work of updating clocks (from HBASE-18395) > -- > > Key: HBASE-18457 > URL: https://issues.apache.org/jira/browse/HBASE-18457 > Project: HBase > Issue Type: Sub-task >Reporter: Appy > > So list of things that need to be considered in future > - Mocking clocks in HMaster/RegionServer to test that clocks get updated on > assign/unassign > - Discuss the idea of adding timestamp type to NodeTime proto --> Get rid of > isLikelyOfType() fn > - In assign/unassign RPCs, either update the clock which matches table's > clock type OR all clocks. Updating only meta clock every time looks kind of > weird right now (and this called out explicitly in comments too). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18542) [HLC] Performance microbenchmarks
Appy created HBASE-18542: Summary: [HLC] Performance microbenchmarks Key: HBASE-18542 URL: https://issues.apache.org/jira/browse/HBASE-18542 Project: HBase Issue Type: Sub-task Reporter: Appy Need tests to benchmark performance of Clock#now() and update() functions (for all types of clocks). If update() is too costly, we can do optimizations in ExecuteProceduresRemoteCall#call() and other places where we call update() in loop. Instead, it might be faster to calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18509) [HLC] Finishing cleanups
Appy created HBASE-18509: Summary: [HLC] Finishing cleanups Key: HBASE-18509 URL: https://issues.apache.org/jira/browse/HBASE-18509 Project: HBase Issue Type: Sub-task Reporter: Appy - Rename classes to more consistent naming: SystemClock, SystemMonotonicClock, HLClock - Move implementations out from Clock interface. It's a simple interface of 6 fns but very overloaded right now with everything put inside it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
Appy created HBASE-18508: Summary: [HLC] Fix timing out tests in HBASE-14070.HLC branch Key: HBASE-18508 URL: https://issues.apache.org/jira/browse/HBASE-18508 Project: HBase Issue Type: Sub-task Reporter: Appy Pre-commit runs for this branch output a huge list of timed out tests. Not seeing those in master branch. Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18499) Design improvements to Clock.java
[ https://issues.apache.org/jira/browse/HBASE-18499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18499. -- Resolution: Duplicate > Design improvements to Clock.java > - > > Key: HBASE-18499 > URL: https://issues.apache.org/jira/browse/HBASE-18499 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > > - Delete PhysicalClock interface which seems useless give we have SYSTEM > implementation of Clock which is basically the same. > - Embed systemMonotonic clock into HLC to get rid of redundancy in management > of physical component, esp logic around ensuring monotonicity. > - Make max_skew in clocks configurable > - Remove isMonotonicallyIncreasing() which is not used. > - update logical overflow test to not use hooks but prod code path > - Added lots of comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18499) Design improvements to Clock.java
Appy created HBASE-18499: Summary: Design improvements to Clock.java Key: HBASE-18499 URL: https://issues.apache.org/jira/browse/HBASE-18499 Project: HBase Issue Type: Sub-task Reporter: Appy Assignee: Appy - Delete PhysicalClock interface which seems useless give we have SYSTEM implementation of Clock which is basically the same. - Embed systemMonotonic clock into HLC to get rid of redundancy in management of physical component, esp logic around ensuring monotonicity. - Make max_skew in clocks configurable - Remove isMonotonicallyIncreasing() which is not used. - update logical overflow test to not use hooks but prod code path - Added lots of comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18498) Design improvements to Clock.java
Appy created HBASE-18498: Summary: Design improvements to Clock.java Key: HBASE-18498 URL: https://issues.apache.org/jira/browse/HBASE-18498 Project: HBase Issue Type: Sub-task Reporter: Appy Assignee: Appy - Delete PhysicalClock interface which seems useless give we have SYSTEM implementation of Clock which is basically the same. - Embed systemMonotonic clock into HLC to get rid of redundancy in management of physical component, esp logic around ensuring monotonicity. - Make max_skew in clocks configurable - Remove isMonotonicallyIncreasing() which is not used. - update logical overflow test to not use hooks but prod code path - Added lots of comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18496) Fix TestTimestampType failing on isLikelyOfType tests
[ https://issues.apache.org/jira/browse/HBASE-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18496. -- Resolution: Fixed > Fix TestTimestampType failing on isLikelyOfType tests > - > > Key: HBASE-18496 > URL: https://issues.apache.org/jira/browse/HBASE-18496 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18496.HBASE-14070.HLC.001.patch > > > The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were > failing after the initial commit of core HLC so this patch fixes that. Note > that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false > if the input timestamp is from before the first half of the first day of 1970. > Changes > * Offset date by second day of year > * Fixed instance of not properly setting ZonedDateTime in > TestTimestampType#testPhysicalIsLikelyOfType > * Changed timestamp for year 2016 in TimestampType to be UTC+0 instead of > UTC+8 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18457) [HLC] Follow up work of updating clocks (from HBASE-18395)
Appy created HBASE-18457: Summary: [HLC] Follow up work of updating clocks (from HBASE-18395) Key: HBASE-18457 URL: https://issues.apache.org/jira/browse/HBASE-18457 Project: HBase Issue Type: Sub-task Reporter: Appy So list of things that need to be considered in future - Mocking clocks in HMaster/RegionServer to test that clocks get updated on assign/unassign - Discuss the idea of adding timestamp type to NodeTime proto --> Get rid of isLikelyOfType() fn - In assign/unassign RPCs, either update the clock which matches table's clock type OR all clocks. Updating only meta clock every time looks kind of weird right now (and this called out explicitly in comments too). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18450) Add test for HBASE-18247
Appy created HBASE-18450: Summary: Add test for HBASE-18247 Key: HBASE-18450 URL: https://issues.apache.org/jira/browse/HBASE-18450 Project: HBase Issue Type: Task Reporter: Appy Assignee: huaxiang sun ref: https://issues.apache.org/jira/browse/HBASE-18247?focusedCommentId=16100472=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16100472 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18432) Prevent clock from getting stuck after update()
Appy created HBASE-18432: Summary: Prevent clock from getting stuck after update() Key: HBASE-18432 URL: https://issues.apache.org/jira/browse/HBASE-18432 Project: HBase Issue Type: Sub-task Reporter: Appy Assignee: Appy There were a [bunch of problems|https://issues.apache.org/jira/browse/HBASE-14070?focusedCommentId=16094013=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16094013] (also copied below) with clock getting stuck after call to update() until it's own system time caught up. Proposed solution is, keeping track of skew separately. - PT = physical time, LT = logical time, ST = system time, X = don't care terms Note that in current implementation, we are passing master clock to RS in open/close region request and RS clock to master in the responses. And they both update their own time on receiving these request/response. Also, on receiving a clock ahead of its own, they update their own clock to its PT+LT, and keep increasing LT till their own ST catches that PT. Problem 1: Logical time window too small. RS clock (10, X) Master clock (20, X) Master --request-> RS RS clock (20, X) While RS's physical java clock (which is backing up physical component of hlc clock) will still take 10 sec to catch up, we'll keep incrementing logical component. That means, in worst case, our logical clock window should be big enough to support all the events that can happen in max skew time. The problem is, that doesn't seem to be the case. Our logical window is 1M events (20bits) and max skew time is 30 sec, that results in 33k max write qps, which is quite low. We can easily see 150k update qps per beefy server with 1k values. Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew time to support ~420k max events per second in worst case clock skew. Problem 2: Cascading logical time increment. When more RS are involved say - 3 RS and 1 master. Let's say max skew is 30 sec. HLC Clocks (physical time, logical time): X = don't care RS1: (50, 100k) Master: (40, X) RS2: (30, X) RS3: (20, X) [RS3's ST behind RS1's by 30 sec.] RS1 replies to master, sends it's clock (50,X). Master's clock (50, X). It'll be another 10 sec before it's own physical clock reaches 50, so HLC's PT will remain 50 for next 10 sec. Master --> RS2 RS2's clock = (50, X). RS2 keeps incrementing LT on writes (since it's own PT is behind) for few seconds before it replies back to master with (50, X+ few 100k). Master's clock = (50, X+ few 100k) [Since master's physical clock hasn't caught up yet, note that it was 10 seconds behind, PT remains 50.]. Master --> RS3 RS3's clock (50, X+few 100k) But RS3's ST is behind RS1's ST by 30 sec, which means it'll keep incrementing LT for next 30 sec (unless it gets a newer clock from master). But the problem is, RS3 has much smaller LT window than actual 1M!! — Problem 3: Single bad RS clock crashing the cluster: If a single RS's clock is bad and a bit faster, it'll catch time and keep pulling master's PT with it. If 'real time' is say 20, max skew time is 10, and bad RS is at time 29.9, it'll pull master to 29.9 (via next response), and then any RS less than 19.9, i.e. just 0.1 sec away from real time will die due to higher than max skew. This can bring whole clusters down! — Problem 4: Time jumps (not a bug, but more of a nuisance) Say a RS is behind master by 20 sec. On each communication from master, RS will update its own PT to master's PT, and it'll remain that till RS's ST catches up. If there are frequent communication from master, ST might never catch up and RS's PT will actually look like discrete time jumps rather than continuous time. For eg. If master communicated with RS at times 30, 40, 50 (RSs corresponding times are 10, 20, 30), than all events on RS between time [10, 50] will be timestamped with either 30, 40 or 50. — -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18385) Enable HLC for just the meta table
[ https://issues.apache.org/jira/browse/HBASE-18385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18385. -- Resolution: Fixed > Enable HLC for just the meta table > -- > > Key: HBASE-18385 > URL: https://issues.apache.org/jira/browse/HBASE-18385 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Attachments: HBASE-18385.HBASE-14070.HLC.001.patch > > > This task covers the patch for enabling HLC for the meta table while keeping > the remaining tables by default use the system clock. The patch for this task > will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. > Credit for this work goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18328) Undoing using the master's timestamp for meta updates
[ https://issues.apache.org/jira/browse/HBASE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18328. -- Resolution: Fixed > Undoing using the master's timestamp for meta updates > - > > Key: HBASE-18328 > URL: https://issues.apache.org/jira/browse/HBASE-18328 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: > 0001-HBASE-14070-Undoing-the-use-of-master-s-timestamp-fo.patch, > HBASE-18328.HBASE-14070.HLC.001.patch, HBASE-18328.HBASE-14070.HLC.002.patch > > > After introducing [the Core HLC > framework|https://issues.apache.org/jira/browse/HBASE-18305], the next step > is to remove all instances where the timestamp is explicitly set for meta > updates. Most of these changes take place in MetaTableAccessor. The patch for > this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch. > Credit for this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18341) Update findHangingTests.py script to match changed consoleText of trunk build
[
https://issues.apache.org/jira/browse/HBASE-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy resolved HBASE-18341.
--
Resolution: Fixed
> Update findHangingTests.py script to match changed consoleText of trunk build
> -
>
> Key: HBASE-18341
> URL: https://issues.apache.org/jira/browse/HBASE-18341
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Fix For: 2.0.0, 3.0.0
>
> Attachments: patch
>
>
> consoleText in earlier builds used to report test results as shown below
> {noformat}
> Running org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.753 s - in
> org.apache.hadoop.hbase.client.TestSizeFailures
> {noformat}
> However now we have "[INFO]" / "[WARNING]" in the beginning of lines which
> messed up regex matcher. Updating regex matches to be more general.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18341) Update findHangingTests.py script to match changed consoleText of trunk build
Appy created HBASE-18341:
Summary: Update findHangingTests.py script to match changed
consoleText of trunk build
Key: HBASE-18341
URL: https://issues.apache.org/jira/browse/HBASE-18341
Project: HBase
Issue Type: Bug
Reporter: Appy
Assignee: Appy
consoleText in earlier builds used to report test results as shown below
{noformat}
Running org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 47.753 s - in
org.apache.hadoop.hbase.client.TestSizeFailures
{noformat}
However now we have "[INFO]" / "[WARNING]" in the beginning of lines which
messed up regex matcher. Updating regex matches to be more general.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-18076) Flaky dashboard improvement: Add status markers to show trends of failure/success
[ https://issues.apache.org/jira/browse/HBASE-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18076. -- Resolution: Fixed > Flaky dashboard improvement: Add status markers to show trends of > failure/success > - > > Key: HBASE-18076 > URL: https://issues.apache.org/jira/browse/HBASE-18076 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: dashboard.html, HBASE-18076.master.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18076) Flaky dashboard improvement: Add status markers to show trends of failure/success
Appy created HBASE-18076: Summary: Flaky dashboard improvement: Add status markers to show trends of failure/success Key: HBASE-18076 URL: https://issues.apache.org/jira/browse/HBASE-18076 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18068) Fix flaky test TestAsyncSnapshotAdminApi
Appy created HBASE-18068:
Summary: Fix flaky test TestAsyncSnapshotAdminApi
Key: HBASE-18068
URL: https://issues.apache.org/jira/browse/HBASE-18068
Project: HBase
Issue Type: Sub-task
Reporter: Appy
Assignee: Appy
Test failures
{noformat}
org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi.testRestoreSnapshot
org.apache.hadoop.hbase.snapshot.RestoreSnapshotException:
org.apache.hadoop.hbase.snapshot.RestoreSnapshotException: Restore already in
progress on the table=testRestoreSnapshot
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.restoreSnapshot(SnapshotManager.java:854)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.restoreSnapshot(SnapshotManager.java:818)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.restoreOrCloneSnapshot(SnapshotManager.java:780)
at org.apache.hadoop.hbase.master.HMaster$14.run(HMaster.java:2324)
at
org.apache.hadoop.hbase.master.procedure.MasterProcedureUtil.submitProcedure(MasterProcedureUtil.java:133)
at org.apache.hadoop.hbase.master.HMaster.restoreSnapshot(HMaster.java:2320)
at
org.apache.hadoop.hbase.master.MasterRpcServices.restoreSnapshot(MasterRpcServices.java:1224)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257)
org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi.testDeleteSnapshots
org.apache.hadoop.hbase.snapshot.SnapshotCreationException:
org.apache.hadoop.hbase.snapshot.SnapshotCreationException: Rejected taking {
ss=snapshotName1 table=testDeleteSnapshots type=FLUSH } because we are already
running another snapshot on the same table { ss=snapshotName1
table=testDeleteSnapshots type=FLUSH }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:440)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:497)
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:598)
at
org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1299)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257)
org.apache.hadoop.hbase.client.TestAsyncSnapshotAdminApi.testListSnapshots
org.apache.hadoop.hbase.snapshot.SnapshotDoesNotExistException: Snapshot
'snapshotName2' doesn't exist on the filesystem
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.deleteSnapshot(SnapshotManager.java:289)
at
org.apache.hadoop.hbase.master.MasterRpcServices.deleteSnapshot(MasterRpcServices.java:461)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:257)
{noformat}
https://builds.apache.org/job/HBASE-Flaky-Tests/16152/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HBASE-18044) Bug fix in flaky-dashboard-template
Appy created HBASE-18044: Summary: Bug fix in flaky-dashboard-template Key: HBASE-18044 URL: https://issues.apache.org/jira/browse/HBASE-18044 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy Priority: Minor Due to some scoping issues, counters don't work in jinja templates like we expect them to (in java/c++,etc). (http://stackoverflow.com/questions/7537439/how-to-increment-a-variable-on-a-for-loop-in-jinja-template). Due to this, clicking show/hide button for tests which fail in multiple urls will not work sometime. Fixing it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17732) Add CpType template to CoprocessorEnvironment
Appy created HBASE-17732: Summary: Add CpType template to CoprocessorEnvironment Key: HBASE-17732 URL: https://issues.apache.org/jira/browse/HBASE-17732 Project: HBase Issue Type: Improvement Reporter: Appy Assignee: Appy Adding template for coprocessor type to CoprocessorEnvironment will: - get rid of 'instanceOf' checks we do in various coprocessor hosts - allow us to load only relevant coprocessors in hosts (right now i think each host loads all coprocs and it's only during execOperation that it checks if the coproc is of correct type ie XXCoprocessorHost will load XXObserver, YYObserver, and all others, and in will check in execOperation if coproc is of type XXObserver and ignore the rest). - allow sharing of a bunch functions/classes which are currently 99% duplicate of each other. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations(). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17730) Migration to 2.0 for coprocessors
Appy created HBASE-17730: Summary: Migration to 2.0 for coprocessors Key: HBASE-17730 URL: https://issues.apache.org/jira/browse/HBASE-17730 Project: HBase Issue Type: Sub-task Reporter: Appy Jiras breaking coprocessor compatibility should be marked with component ' Coprocessor', and label 'incompatible'. Close to releasing 2.0, we should go through all such jiras and write down steps for migrating coprocessor easily. The idea is, it might be very hard to fix coprocessor breakages by reverse engineering errors, but will be easier we suggest easiest way to fix breakages resulting from each individual incompatible change. For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors because BaseXXXObserver classes are gone, but the fix is actually just one line : simply change "Foo extends BaseXXXObserver" to "Foo implements XXXObserver". -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-17699) Fix TestLockProcedure
Appy created HBASE-17699: Summary: Fix TestLockProcedure Key: HBASE-17699 URL: https://issues.apache.org/jira/browse/HBASE-17699 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy TestLockProcedure is failing consistently after HBASE-17605. It's interesting that HadoopQA didn't report any test failures on that jira. Anyways, need to fix the test now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
