[jira] [Updated] (HBASE-18807) Remove PB references from Observers for Quotas
[
https://issues.apache.org/jira/browse/HBASE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-18807:
---
Attachment: HBASE-18807.003.branch-2.patch
.003 Lots of cleanup. Some missing test annotations. Marked
{{GlobalQuotaSettings}} as {{LimitedPrivate(COPROC)}}. Better javadoc.
> Remove PB references from Observers for Quotas
> --
>
> Key: HBASE-18807
> URL: https://issues.apache.org/jira/browse/HBASE-18807
> Project: HBase
> Issue Type: Sub-task
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18807.001.branch-2.patch,
> HBASE-18807.002.branch-2.patch, HBASE-18807.003.branch-2.patch
>
>
> Break-out from the parent:
> Same idea, just applied to the Observer methods for pre/post quota
> operations. Requires changes to MasterQuotaManager and the QuotaSettings
> implementations as some business logic is written on the PB objects directly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175472#comment-16175472 ] stack commented on HBASE-18825: --- Oh, yeah, as suggested above, needs airing on dev list if only to bring folks attention to the tendency. > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175511#comment-16175511 ] Andrew Purtell commented on HBASE-12081: I also added branch-1 and branch-1.4, but I fear that may be optimistic. Will drop branch-1.4 if need be. We can keep this as a blocker for continuing forward with branch-1 (1.5, etc.) > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[ https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18477: -- Attachment: HBase Read-Replica Clusters Scope doc_v2.pdf Adding the v2 doc as a PDF. [~busbey] sorry for the delay. > Umbrella JIRA for HBase Read Replica clusters > - > > Key: HBASE-18477 > URL: https://issues.apache.org/jira/browse/HBASE-18477 > Project: HBase > Issue Type: New Feature >Reporter: Zach York >Assignee: Zach York > Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase > Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope > doc_v2.docx, HBase Read-Replica Clusters Scope doc_v2.pdf > > > Recently, changes (such as HBASE-17437) have unblocked HBase to run with a > root directory external to the cluster (such as in Amazon S3). This means > that the data is stored outside of the cluster and can be accessible after > the cluster has been terminated. One use case that is often asked about is > pointing multiple clusters to one root directory (sharing the data) to have > read resiliency in the case of a cluster failure. > > This JIRA is an umbrella JIRA to contain all the tasks necessary to create a > read-replica HBase cluster that is pointed at the same root directory. > > This requires making the Read-Replica cluster Read-Only (no metadata > operation or data operations). > Separating the hbase:meta table for each cluster (Otherwise HBase gets > confused with multiple clusters trying to update the meta table with their ip > addresses) > Adding refresh functionality for the meta table to ensure new metadata is > picked up on the read replica cluster. > Adding refresh functionality for HFiles for a given table to ensure new data > is picked up on the read replica cluster. > > This can be used with any existing cluster that is backed by an external > filesystem. > > Please note that this feature is still quite manual (with the potential for > automation later). > > More information on this particular feature can be found here: > https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-12081: --- Fix Version/s: 1.5.0 1.4.0 > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18127) Enable state to be passed between the region observer coprocessor hook calls
[ https://issues.apache.org/jira/browse/HBASE-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175538#comment-16175538 ] Andrew Purtell commented on HBASE-18127: RB link is https://reviews.apache.org/r/62432/ [~abhishek.chouhan] in the future you can add the group 'hbase' to the RB review so everyone with interest in the project gets notifications. > Enable state to be passed between the region observer coprocessor hook calls > > > Key: HBASE-18127 > URL: https://issues.apache.org/jira/browse/HBASE-18127 > Project: HBase > Issue Type: New Feature >Reporter: Lars Hofhansl >Assignee: Abhishek Singh Chouhan > Attachments: HBASE-18127.master.001.patch, > HBASE-18127.master.002.patch, HBASE-18127.master.002.patch, > HBASE-18127.master.003.patch > > > Allow regionobserver to optionally skip postPut/postDelete when > postBatchMutate was called. > Right now a RegionObserver can only statically implement one or the other. In > scenarios where we need to work sometimes on the single postPut and > postDelete hooks and sometimes on the batchMutate hooks, there is currently > no place to convey this information to the single hooks. I.e. the work has > been done in the batch, skip the single hooks. > There are various solutions: > 1. Allow some state to be passed _per operation_. > 2. Remove the single hooks and always only call batch hooks (with a default > wrapper for the single hooks). > 3. more? > [~apurtell], what we had discussed a few days back. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1617#comment-1617 ] Mike Drob commented on HBASE-12081: --- I had looked at this before and the first big blockers that I found were that the version of findbugs we use was not compatible (I think because the underlying asm needs to be re-instrumented for Java 9) and that scala needed an update (so hbase-spark has issues). Commenting those bits out, we could at least compile, but I never finished getting tests to pass. > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174998#comment-16174998 ] Ted Yu edited comment on HBASE-18651 at 9/22/17 12:16 AM: -- [~md...@cloudera.com]: Do you want to take one more look ? was (Author: yuzhih...@gmail.com): Planning to commit later today if there is no further review comment. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175700#comment-16175700
]
Vladimir Rodionov commented on HBASE-18843:
---
This method I need to overwrite
{code}
private Path computeSourceRootPath(FileStatus sourceStatus,
DistCpOptions options) throws IOException {
{code}
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch, HBASE-18843-v2.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175741#comment-16175741 ] stack commented on HBASE-18298: --- Yeah, name CoprocessorRSServices as CompressorRegionServerServices and method name should be getCompressorRegionServerServices How comes getRpcServer? Some CPs need this? I'd think CP is running in a RS. We are already inside past the RPC. Why would a CP be meddling w/ RPC? Naughty CP! Yeah, almost ditto for getFileSystem though I suppose they might want to get to HDFS. Above I question this being in the Interface. Now I am fine w/ it. Man, what is this for getRecoveringRegions ? Seems totally internal. Used by our recover regions handler. CPs shouldn't be looking at this? Yeah, you might just extend OnlineRegions and throw IllegalAccess if they try to change the Region Map. It has a few of the methods that you have in here. Then it also extends Server which is at root of RegionServerService and it also has some of the methods you have in here plus a few other innocuous ones -- We'd do this so we don't have yet another Interface to explain. SecureBulkLoadEndpoint is deprecated and going away, becoming integral, so you don't have to worry about the hack you have going on in here (smile). I think the patch is starting to look good. > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175419#comment-16175419
]
Hadoop QA commented on HBASE-18796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
37m 29s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18796 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888357/HBASE-18796-addendum.master.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 70f172fa95a2 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e393599 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8726/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8726/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175470#comment-16175470 ] stack commented on HBASE-18825: --- Ok. Thanks for the philosophy. That helps. So, we disabuse ourselves of any notion that another might want to put in place their own HRegion implemenation -- whether a subclass or a new implementation altogether -- if only because we've never done the work to ensure it is supported? I'm good w/ that (+1 on cleanup of "hbase.hregion.impl" that [~anoop.hbase] helpfully turned up). +1 too on using implementation internally rather than Interfaces. Going via Interfaces which are NOT IA.Private means we limit our ability to change. I like this observation. We need to put it up on the hbase dev billboard (I can add to dev section in guide at least). We make same call for entities below HRegion so, it is not possible to provide an alternate Store? I'm good with that. What about StoreFile? I'm thinking about alternate HFile implementations. What if someone wants to plug in a columnar-based file format? Or we want to do a V4 of HFile. This is harder for me to swallow. Looking at your patch though, I see that you do not disturb StoreFileReader nor StoreFileWriter. So it seems like alternate HFile implementations are still possible? If so, good. That Region and Store were made to feed CP is also an interesting observation. It seems like Region gets pretty wide usage in the codebase since its introduction, beyond CP-only use. No harm I suppose. So, I buy-in but need clarification around new HFile implementations. I'd also like to help message this philosophy so it sees adoption beyond [~Apache9] contribs. > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.003.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rich Howarth updated HBASE-18847:
-
Status: Patch Available (was: Open)
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18775: -- Attachment: HBASE-18775.HBASE-18477.003.patch > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch, HBASE-18775.HBASE-18477.003.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175679#comment-16175679 ] Ted Yu commented on HBASE-18843: Can you list which private methods you depend on ? > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch, HBASE-18843-v2.patch > > > Currently, we copy bulk loaded files to backup one-by-one on a client side > (where backup create runs). This has to be replaced with DistCp copying. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175677#comment-16175677 ] Vladimir Rodionov commented on HBASE-18843: --- Created RB: https://reviews.apache.org/r/62486/ > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch, HBASE-18843-v2.patch > > > Currently, we copy bulk loaded files to backup one-by-one on a client side > (where backup create runs). This has to be replaced with DistCp copying. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175461#comment-16175461 ] Zach York commented on HBASE-18840: --- [~stack] Yes, I am developing against branch-1 (more specifically 1.3.1, but it is similar enough). In the near future, I expect I will start deving against branch-2. Since the other issue was resolved, I'll just put up a patch and you can look at the specific implementation. It is possible some of it doesn't make sense with newer features in branch-2, but we can discuss that there. > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Status: Patch Available (was: Open) > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[ https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-18843: -- Attachment: HBASE-18843-v2.patch v2 (added interface annotation to the new class) > Add DistCp support to incremental backup with bulk loading > -- > > Key: HBASE-18843 > URL: https://issues.apache.org/jira/browse/HBASE-18843 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18843-v1.patch, HBASE-18843-v2.patch > > > Currently, we copy bulk loaded files to backup one-by-one on a client side > (where backup create runs). This has to be replaced with DistCp copying. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18807) Remove PB references from Observers for Quotas
[
https://issues.apache.org/jira/browse/HBASE-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175698#comment-16175698
]
Hadoop QA commented on HBASE-18807:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
8s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
57s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 53s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
26s{color} | {color:red} hbase-server generated 4 new + 0 unchanged - 0 fixed =
4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
36s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m
9s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}163m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | instanceof will always return true for all non-null values in
org.apache.hadoop.hbase.quotas.MasterQuotaManager$1.update(GlobalQuotaSettings),
since all org.apache.hadoop.hbase.quotas.GlobalQuotaSettings are instances of
org.apache.hadoop.hbase.quotas.GlobalQuotaSettings At
MasterQuotaManager.java:for all non-null values in
org.apache.hadoop.hbase.quotas.MasterQuotaManager$1.update(GlobalQuotaSettings),
since all org.apache.hadoop.hbase.quotas.GlobalQuotaSettings are instances of
org.apache.hadoop.hbase.quotas.GlobalQuotaSettings At
MasterQuotaManager.java:[line 162] |
| |
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Affects Version/s: HBASE-18477 > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch, HBASE-18840.HBASE-18477.003.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175496#comment-16175496 ] Dave Latham commented on HBASE-12081: - Java 9 is now GA > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.001.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rich Howarth updated HBASE-18847:
-
Attachment: HBASE-18847.v1.patch
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17732) Coprocessor Design Improvements
[
https://issues.apache.org/jira/browse/HBASE-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175525#comment-16175525
]
Josh Elser commented on HBASE-17732:
Will try to give you a heads-up assuming that HBASE-18807 lands before you get
this one in, [~appy].
Should be a simple conflict to resolve by switching the {{QuotaProtos.Quotas}}
object with the new object {{GlobalQuotaSettings}} (assuming the class name
doesn't change).
> Coprocessor Design Improvements
> ---
>
> Key: HBASE-17732
> URL: https://issues.apache.org/jira/browse/HBASE-17732
> Project: HBase
> Issue Type: Improvement
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-17732.master.001.patch,
> HBASE-17732.master.002.patch, HBASE-17732.master.003.patch,
> HBASE-17732.master.004.patch
>
>
> The two main changes are:
> * *Adding template for coprocessor type to CoprocessorEnvironment i.e.
> {{interface CoprocessorEnvironment}}*
> ** Enables us to load only relevant coprocessors in hosts. Right now each
> type of host loads all types of coprocs and it's only during execOperation
> that it checks if the coproc is of correct type i.e. XCoprocessorHost will
> load XObserver, YObserver, and all others, and will check in execOperation if
> {{coproc instanceOf XObserver}} and ignore the rest.
> ** Allow sharing of a bunch functions/classes which are currently
> duplicated in each host. For eg. CoprocessorOperations,
> CoprocessorOperationWithResult, execOperations().
> * *Introduce 4 coprocessor classes and use composition between these new
> classes and and old observers*
> ** The real gold here is, moving forward, we'll be able to break down giant
> everything-in-one observers (masterobserver has 100+ functions) into smaller,
> more focused observers. These smaller observer can then have different compat
> guarantees!!
> Here's a more detailed design doc:
> https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18861) [C++] Use boost::optional instead of std::experimental::optional
[
https://issues.apache.org/jira/browse/HBASE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175675#comment-16175675
]
marco polo commented on HBASE-18861:
+1 -- Patch looks good and works on local platforms that don't have C++17
support to include Centos 7.3 and Mac Sierra.
> [C++] Use boost::optional instead of std::experimental::optional
>
>
> Key: HBASE-18861
> URL: https://issues.apache.org/jira/browse/HBASE-18861
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-18861_v1.patch
>
>
> Our Marc Parisi indicates that using std::experimental is not prefered in
> production code, and causes compilation problems especially for compilers
> which do not have C++17 support.
> Our usage of {{std::experimental::optional}} is very small and can be easily
> replaced with {{boost::optional}}. We depend on boost anyways for other
> reasons.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[
https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175719#comment-16175719
]
Hadoop QA commented on HBASE-18775:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s{color} | {color:green} HBASE-18477 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} HBASE-18477 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} HBASE-18477 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s{color} | {color:green} HBASE-18477 passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 5m
39s{color} | {color:red} branch has 13 errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} HBASE-18477 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} HBASE-18477 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 4m
4s{color} | {color:red} patch has 13 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
39m 50s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 2s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.io.TestHeapSize |
| | hadoop.hbase.TestCheckTestClasses |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18775 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888383/HBASE-18775.HBASE-18477.002.patch
|
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b17f2c5f0aac 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
[jira] [Commented] (HBASE-18817) Pull hbase-spark module out of branch-2
[ https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175722#comment-16175722 ] stack commented on HBASE-18817: --- +1 Didn't even look at the patch. Couldn't (Tearing up). > Pull hbase-spark module out of branch-2 > --- > > Key: HBASE-18817 > URL: https://issues.apache.org/jira/browse/HBASE-18817 > Project: HBase > Issue Type: Task > Components: spark >Affects Versions: 2.0.0-alpha-2 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18817-branch-2.v0.patch > > > see DISCUSS here: > https://s.apache.org/UJAf > Sadly, feature is slipping out of branch-2. We can work out inclusion for > downstream once we have some inertia again. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175593#comment-16175593 ] Zach York commented on HBASE-18775: --- [~tedyu] The latest patch and rb addresses your comments. > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175659#comment-16175659 ] stack commented on HBASE-12081: --- Thanks [~mdrob] > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0, 1.4.0, 1.5.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175680#comment-16175680 ] Hudson commented on HBASE-18731: FAILURE: Integrated in Jenkins build HBase-2.0 #554 (See [https://builds.apache.org/job/HBase-2.0/554/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (busbey: rev c1f5122fab9d5fec52a7346ef71e0776fc3180f6) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18840) Add functionality to refresh meta table at master startup
[ https://issues.apache.org/jira/browse/HBASE-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18840: -- Attachment: HBASE-18840.HBASE-18477.002.patch > Add functionality to refresh meta table at master startup > - > > Key: HBASE-18840 > URL: https://issues.apache.org/jira/browse/HBASE-18840 > Project: HBase > Issue Type: Sub-task >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18840.HBASE-18477.001.patch, > HBASE-18840.HBASE-18477.002.patch > > > If a HBase cluster’s hbase:meta table is deleted or a cluster is started with > a new meta table, HBase needs the functionality to synchronize it’s metadata > from Storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18775: -- Affects Version/s: HBASE-18477 > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18775: -- Status: Patch Available (was: In Progress) > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175669#comment-16175669
]
Vladimir Rodionov edited comment on HBASE-18843 at 9/21/17 11:42 PM:
-
{quote}
Have you considered using other format which is more amiable to upgrade ?
{quote}
That (SequenceFile) is format DistCp understands and expects.
was (Author: vrodionov):
{quote}
Have you considered using other format which is more amiable to upgrade ?
{quote}
That is format DistCp understands and expects.
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18817) Pull hbase-spark module out of branch-2
[
https://issues.apache.org/jira/browse/HBASE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175688#comment-16175688
]
Hadoop QA commented on HBASE-18817:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
0s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
50s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
52s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
29s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-spark-it hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
0s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} scaladoc {color} | {color:green} 2m
4s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} scalac {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
0s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 12s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 27s{color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hbase-assembly . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} hbase-assembly in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} root generated 0 new + 2 unchanged - 23 fixed = 2
total (was 25) {color} |
| {color:green}+1{color} |
[jira] [Commented] (HBASE-18183) Region interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175757#comment-16175757 ] stack commented on HBASE-18183: --- Want to take a look [~anoop.hbase] ? What were you considering when you were going through the Region Interface? What were you looking for? (From our conversation a while back I have) * All Observers, make sure param and return are NOT private... fix if they are. * Change HTD to TD * LockProcedure is Private. * Go through and stuff that is Private... should not be in CP Interface. * CompactionRequest make into an Interface. * Should be Interfaces everywhere. ... > Region interface cleanup for CP expose > -- > > Key: HBASE-18183 > URL: https://issues.apache.org/jira/browse/HBASE-18183 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175457#comment-16175457
]
Hadoop QA commented on HBASE-18796:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
46s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
29s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
39s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
20m 40s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
3s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:6f1cc2c |
| JIRA Issue | HBASE-18796 |
| JIRA Patch URL |
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12081: -- Fix Version/s: 2.0.0 > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175499#comment-16175499 ] stack commented on HBASE-12081: --- Making this a blocker for 2.0.0. Ensure we at least run on a jdk9. > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18775: -- Attachment: HBASE-18775.HBASE-18477.002.patch > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175669#comment-16175669
]
Vladimir Rodionov commented on HBASE-18843:
---
{quote}
Have you considered using other format which is more amiable to upgrade ?
{quote}
That is format DistCp understands and expects.
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[ https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18775: -- Attachment: HBASE-18775.HBASE-18477.004.patch > Add a Global Read-Only property to turn off all writes for the cluster > -- > > Key: HBASE-18775 > URL: https://issues.apache.org/jira/browse/HBASE-18775 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18775.HBASE-18477.001.patch, > HBASE-18775.HBASE-18477.002.patch, HBASE-18775.HBASE-18477.003.patch, > HBASE-18775.HBASE-18477.004.patch > > > As part of HBASE-18477, we need a way to turn off all modification for a > cluster. This patch extends the read only mode used by replication to disable > all data and metadata operations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18843) Add DistCp support to incremental backup with bulk loading
[
https://issues.apache.org/jira/browse/HBASE-18843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175665#comment-16175665
]
Vladimir Rodionov commented on HBASE-18843:
---
{quote}
Can you do some refactoring ?
{quote}
Does not seem feasible. SimpleCopyListing has some private methods I need
access to.
> Add DistCp support to incremental backup with bulk loading
> --
>
> Key: HBASE-18843
> URL: https://issues.apache.org/jira/browse/HBASE-18843
> Project: HBase
> Issue Type: Improvement
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18843-v1.patch
>
>
> Currently, we copy bulk loaded files to backup one-by-one on a client side
> (where backup create runs). This has to be replaced with DistCp copying.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-12081) Considering Java 9
[ https://issues.apache.org/jira/browse/HBASE-12081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12081: -- Priority: Blocker (was: Major) > Considering Java 9 > -- > > Key: HBASE-12081 > URL: https://issues.apache.org/jira/browse/HBASE-12081 > Project: HBase > Issue Type: Umbrella >Reporter: Andrew Purtell >Priority: Blocker > Fix For: 2.0.0 > > > Java 9 will ship in 2016. This will be the first Java release that makes a > significant compatibility departure from earlier runtimes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175501#comment-16175501 ] Andrew Purtell commented on HBASE-18796: I'm running full test suites for HBASE-18786. I have rolled in the addendums as separate commits in those working branches so will test them both together. If good I will commit them. Thanks for the addendum [~abhishek.chouhan] and thanks for pointing out the issue [~ted_yu] > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796-addendum.branch-1.patch, > HBASE-18796-addendum.master.patch, HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18847) remove unneeded synchronized block from hfilev2 warning in branch-1.2
[
https://issues.apache.org/jira/browse/HBASE-18847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175515#comment-16175515
]
Hadoop QA commented on HBASE-18847:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color}
| {color:red} HBASE-18847 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.4.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-18847 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888375/HBASE-18847.v1.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8730/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> remove unneeded synchronized block from hfilev2 warning in branch-1.2
> -
>
> Key: HBASE-18847
> URL: https://issues.apache.org/jira/browse/HBASE-18847
> Project: HBase
> Issue Type: Bug
> Components: Performance
>Affects Versions: 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6
>Reporter: Rich Howarth
>Assignee: Rich Howarth
>Priority: Critical
> Fix For: 1.2.7
>
> Attachments: HBASE-18847.v1.patch
>
>
> The below code block starts at line 277 of HFileWriterV2.java. Class-level
> synchronization in a heavily used code path has a demonstrably significant
> negative effect on performance. I tested forcing a major compaction with 18
> compaction threads per node; removing the synchronization resulted in an
> order of magnitude performance increase, with the bottleneck then being at
> the disks (where I want it to be).
> {code:java}
> synchronized (HFileWriterV2.class) {
> if (WARN_CELL_WITH_TAGS && getFileContext().isIncludesTags()) {
> LOG.warn("A minimum HFile version of " +
> HFile.MIN_FORMAT_VERSION_WITH_TAGS
> + " is required to support cell attributes/tags. Consider setting "
> + HFile.FORMAT_VERSION_KEY + " accordingly.");
> WARN_CELL_WITH_TAGS = false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18773) Add utility to determine if a TableName is a meta table
[ https://issues.apache.org/jira/browse/HBASE-18773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175536#comment-16175536 ] Zach York commented on HBASE-18773: --- [~stack] I created HBASE-18860 to address where to move this file (and where to put all of the new ReadReplica files). > Add utility to determine if a TableName is a meta table > --- > > Key: HBASE-18773 > URL: https://issues.apache.org/jira/browse/HBASE-18773 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York > Fix For: HBASE-18477 > > Attachments: > 0001-HBASE-18773-Add-utility-method-to-determine-if-a-Tab.patch, > HBASE-18773.HBASE-18477.001.patch, HBASE-18773.HBASE-18477.002.patch, > HBASE-18773.HBASE-18477.003.patch, HBASE-18773.HBASE-18477.004.patch, > HBASE-18773.HBASE-18477.005.patch, HBASE-18773.HBASE-18477.006.patch > > > HBASE-18444 adds a method of specifying a meta table suffix. To continue work > on HBASE-18477, we need a way to determine if a TableName is a meta table. > This patch adds this method and a unit test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18860) Determine where to move ReadReplicaClustersTableNameUtils
Zach York created HBASE-18860: - Summary: Determine where to move ReadReplicaClustersTableNameUtils Key: HBASE-18860 URL: https://issues.apache.org/jira/browse/HBASE-18860 Project: HBase Issue Type: Sub-task Affects Versions: HBASE-18477 Reporter: Zach York Assignee: Zach York Priority: Minor In HBASE-18773, Stack brought up the (good) point that ReadReplicaClustersTableNameUtils does not really belong in the hbase-common package. This follow-up JIRA is to determine the correct place for Read-Replica code and to move this utils class there. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[
https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175612#comment-16175612
]
Ted Yu commented on HBASE-18775:
{code}
+ private static final TableName TABLE_NAME =
TableName.valueOf("TestGlobalReadOnly");
{code}
Can the test cover custom namespace ?
{code}
+config.setBoolean("hbase.testing.nonamespacemanager", true);
{code}
Why is the above needed ?
> Add a Global Read-Only property to turn off all writes for the cluster
> --
>
> Key: HBASE-18775
> URL: https://issues.apache.org/jira/browse/HBASE-18775
> Project: HBase
> Issue Type: Sub-task
> Components: master, regionserver
>Affects Versions: HBASE-18477
>Reporter: Zach York
>Assignee: Zach York
> Attachments: HBASE-18775.HBASE-18477.001.patch,
> HBASE-18775.HBASE-18477.002.patch
>
>
> As part of HBASE-18477, we need a way to turn off all modification for a
> cluster. This patch extends the read only mode used by replication to disable
> all data and metadata operations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18861) [C++] Use boost::optional instead of std::experimental::optional
[
https://issues.apache.org/jira/browse/HBASE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-18861:
--
Attachment: hbase-18861_v1.patch
> [C++] Use boost::optional instead of std::experimental::optional
>
>
> Key: HBASE-18861
> URL: https://issues.apache.org/jira/browse/HBASE-18861
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Attachments: hbase-18861_v1.patch
>
>
> Our Marc Parisi indicates that using std::experimental is not prefered in
> production code, and causes compilation problems especially for compilers
> which do not have C++17 support.
> Our usage of {{std::experimental::optional}} is very small and can be easily
> replaced with {{boost::optional}}. We depend on boost anyways for other
> reasons.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18859) Purge PB from BulkLoadObserver
[ https://issues.apache.org/jira/browse/HBASE-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18859: -- Assignee: stack Status: Patch Available (was: Open) > Purge PB from BulkLoadObserver > -- > > Key: HBASE-18859 > URL: https://issues.apache.org/jira/browse/HBASE-18859 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18859.master.001.patch > > > Noticed by [~appy] We expose PBs in this CP. Pass POJOs. This is like > [~anoop.hbase] and [~elserj] pb purging that is going on contemporaneously. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18859) Purge PB from BulkLoadObserver
[ https://issues.apache.org/jira/browse/HBASE-18859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18859: -- Attachment: HBASE-18859.master.001.patch > Purge PB from BulkLoadObserver > -- > > Key: HBASE-18859 > URL: https://issues.apache.org/jira/browse/HBASE-18859 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: stack > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18859.master.001.patch > > > Noticed by [~appy] We expose PBs in this CP. Pass POJOs. This is like > [~anoop.hbase] and [~elserj] pb purging that is going on contemporaneously. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175689#comment-16175689 ] Hudson commented on HBASE-18731: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3755 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3755/]) HBASE-18731 [compat 1-2] Mark protected methods of QuotaSettings that (busbey: rev e39359986c4765946cde30da2957324cb7c9705c) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaSettings.java > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch, HBASE-18731-branch-1.v1.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18298) RegionServerServices Interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175708#comment-16175708 ] stack commented on HBASE-18298: --- Pardon. I've been away from this important issue. Let me get back to it by responding to your response [~anoop.hbase]. I like your objection to why we can't use OnlineRegions, because it has write methods. Could we just throw IllegalAccess if a CP tries to add/remove Regions? Or could we insert an immutable OnlineRegions? Or make a subinterface that adds the editing methods? Let me look more at the patch to see if my suggestion, a week later, even makes sense. I like how you are working to not expose levers to CPs such as the #stop method. > RegionServerServices Interface cleanup for CP expose > > > Key: HBASE-18298 > URL: https://issues.apache.org/jira/browse/HBASE-18298 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18298.patch, HBASE-18298_V2.patch, > HBASE-18298_V3.patch, HBASE-18298_V4.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175822#comment-16175822 ] Reid Chan commented on HBASE-18651: --- Thanks for those reviews and suggestions [~tedyu], [~mdrob] > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Fix For: 3.0.0, 2.0.0-alpha-4 > > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175950#comment-16175950 ] ramkrishna.s.vasudevan commented on HBASE-18825: bq.What about StoreFile? I'm thinking about alternate HFile implementations. What if someone wants to plug in a columnar-based file format? Or we want to do a V4 of HFile. This is harder for me to swallow. This question comes because we have StoreFile, Hfile, StorefileREader, HFileReader, StoreFileScanner etc. I think StoreFile is only giving some meta data info about that HFile. So should be good here. >From RB bq.But here I can see initReader() and getReader() removed but closeReader and deleteREader() are there. I think that closeReader should ideally be closeStoreFile() and deleteStoreFile()? I think they need not be there in StoreFile and be exposed to CP. Rest LGTM. +1 > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14247) Separate the old WALs into different regionserver directories
[
https://issues.apache.org/jira/browse/HBASE-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175824#comment-16175824
]
Guanghao Zhang commented on HBASE-14247:
[~stack] ping for reviewing. Do you plan merge this improvement to hbase 2.0
release? Thanks.
> Separate the old WALs into different regionserver directories
> -
>
> Key: HBASE-14247
> URL: https://issues.apache.org/jira/browse/HBASE-14247
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Liu Shaohui
>Assignee: Guanghao Zhang
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-14247.master.001.patch,
> HBASE-14247.master.002.patch, HBASE-14247-v001.diff, HBASE-14247-v002.diff,
> HBASE-14247-v003.diff
>
>
> Currently all old WALs of regionservers are achieved into the single
> directory of oldWALs. In big clusters, because of long TTL of WAL or disabled
> replications, the number of files under oldWALs may reach the
> max-directory-items limit of HDFS, which will make the hbase cluster crashed.
> {quote}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException):
> The directory item limit of /hbase/lgprc-xiaomi/.oldlogs is exceeded:
> limit=1048576 items=1048576
> {quote}
> A simple solution is to separate the old WALs into different directories
> according to the server name of the WAL.
> Suggestions are welcomed~ Thanks
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18825) Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface
[ https://issues.apache.org/jira/browse/HBASE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175924#comment-16175924 ] Anoop Sam John commented on HBASE-18825: bq.What about StoreFile? I'm thinking about alternate HFile implementations. What if someone wants to plug in a columnar-based file format? Or we want to do a V4 of HFile. This is harder for me to swallow. When we need new format or a V4, its the HFileReader and HFileWriter what will have to change. HFile on top of it gives top level APIs for creation of readers writer etc. StoreFile is more on top which is having APIs for traversing through next next Cells etc. So I dont think that we use our impl rather than interface will break any possibility for new HFile version.. Here we have interface so as to pass to CPs. So am +1 > Use HStoreFile instead of StoreFile in our own code base and remove > unnecessary methods in StoreFile interface > -- > > Key: HBASE-18825 > URL: https://issues.apache.org/jira/browse/HBASE-18825 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18825.patch, HBASE-18825-v1.patch, > HBASE-18825-v2.patch, HBASE-18825-v3.patch, HBASE-18825-v3.patch > > > Use generic types to avoid too many casts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18477) Umbrella JIRA for HBase Read Replica clusters
[ https://issues.apache.org/jira/browse/HBASE-18477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18477 started by Zach York. - > Umbrella JIRA for HBase Read Replica clusters > - > > Key: HBASE-18477 > URL: https://issues.apache.org/jira/browse/HBASE-18477 > Project: HBase > Issue Type: New Feature >Reporter: Zach York >Assignee: Zach York > Attachments: HBase Read-Replica Clusters Scope doc.docx, HBase > Read-Replica Clusters Scope doc.pdf, HBase Read-Replica Clusters Scope > doc_v2.docx > > > Recently, changes (such as HBASE-17437) have unblocked HBase to run with a > root directory external to the cluster (such as in Amazon S3). This means > that the data is stored outside of the cluster and can be accessible after > the cluster has been terminated. One use case that is often asked about is > pointing multiple clusters to one root directory (sharing the data) to have > read resiliency in the case of a cluster failure. > > This JIRA is an umbrella JIRA to contain all the tasks necessary to create a > read-replica HBase cluster that is pointed at the same root directory. > > This requires making the Read-Replica cluster Read-Only (no metadata > operation or data operations). > Separating the hbase:meta table for each cluster (Otherwise HBase gets > confused with multiple clusters trying to update the meta table with their ip > addresses) > Adding refresh functionality for the meta table to ensure new metadata is > picked up on the read replica cluster. > Adding refresh functionality for HFiles for a given table to ensure new data > is picked up on the read replica cluster. > > This can be used with any existing cluster that is backed by an external > filesystem. > > Please note that this feature is still quite manual (with the potential for > automation later). > > More information on this particular feature can be found here: > https://aws.amazon.com/blogs/big-data/setting-up-read-replica-clusters-with-hbase-on-amazon-s3/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18860) Determine where to move ReadReplicaClustersTableNameUtils
[ https://issues.apache.org/jira/browse/HBASE-18860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175539#comment-16175539 ] Zach York commented on HBASE-18860: --- My initial thoughts are a new package in hbase-server. If this feature becomes large enough to become it's own module, then we can move it out. > Determine where to move ReadReplicaClustersTableNameUtils > - > > Key: HBASE-18860 > URL: https://issues.apache.org/jira/browse/HBASE-18860 > Project: HBase > Issue Type: Sub-task >Affects Versions: HBASE-18477 >Reporter: Zach York >Assignee: Zach York >Priority: Minor > > In HBASE-18773, Stack brought up the (good) point that > ReadReplicaClustersTableNameUtils does not really belong in the hbase-common > package. This follow-up JIRA is to determine the correct place for > Read-Replica code and to move this utils class there. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18861) [C++] Use boost::optional instead of std::experimental::optional
[
https://issues.apache.org/jira/browse/HBASE-18861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-18861:
--
Summary: [C++] Use boost::optional instead of std::experimental::optional
(was: Use boost::optional instead of std::experimental::optional)
> [C++] Use boost::optional instead of std::experimental::optional
>
>
> Key: HBASE-18861
> URL: https://issues.apache.org/jira/browse/HBASE-18861
> Project: HBase
> Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>
> Our Marc Parisi indicates that using std::experimental is not prefered in
> production code, and causes compilation problems especially for compilers
> which do not have C++17 support.
> Our usage of {{std::experimental::optional}} is very small and can be easily
> replaced with {{boost::optional}}. We depend on boost anyways for other
> reasons.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18861) Use boost::optional instead of std::experimental::optional
Enis Soztutar created HBASE-18861:
-
Summary: Use boost::optional instead of std::experimental::optional
Key: HBASE-18861
URL: https://issues.apache.org/jira/browse/HBASE-18861
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Our Marc Parisi indicates that using std::experimental is not prefered in
production code, and causes compilation problems especially for compilers which
do not have C++17 support.
Our usage of {{std::experimental::optional}} is very small and can be easily
replaced with {{boost::optional}}. We depend on boost anyways for other
reasons.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18775) Add a Global Read-Only property to turn off all writes for the cluster
[
https://issues.apache.org/jira/browse/HBASE-18775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175647#comment-16175647
]
Zach York commented on HBASE-18775:
---
[~tedyu] Can you clarify which File you are talking about for:
bq. + private static final TableName TABLE_NAME =
TableName.valueOf("TestGlobalReadOnly");
I changed TestGlobalReadOnly to try to create a non-default namespace. Perhaps
if you have more comments on this, it would be better to do it in RB.
bq. +config.setBoolean("hbase.testing.nonamespacemanager", true);
bq. Why is the above needed ?
This is actually unnecessary now. In branch-1 there was a NamespaceManager that
would fail to initialize if the default and system namespaces weren't created
and couldn't be created. This is not an issue anymore. I have updated the patch
to reflect this.
> Add a Global Read-Only property to turn off all writes for the cluster
> --
>
> Key: HBASE-18775
> URL: https://issues.apache.org/jira/browse/HBASE-18775
> Project: HBase
> Issue Type: Sub-task
> Components: master, regionserver
>Affects Versions: HBASE-18477
>Reporter: Zach York
>Assignee: Zach York
> Attachments: HBASE-18775.HBASE-18477.001.patch,
> HBASE-18775.HBASE-18477.002.patch, HBASE-18775.HBASE-18477.003.patch
>
>
> As part of HBASE-18477, we need a way to turn off all modification for a
> cluster. This patch extends the read only mode used by replication to disable
> all data and metadata operations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18183) Region interface cleanup for CP expose
[ https://issues.apache.org/jira/browse/HBASE-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175758#comment-16175758 ] Umesh Agashe commented on HBASE-18183: -- Hi [~anoop.hbase], Can we deprecate RowProcessor as well? If CP hooks are called, do we need separate RowProcessor hooks called by processRowsWithLocks(). > Region interface cleanup for CP expose > -- > > Key: HBASE-18183 > URL: https://issues.apache.org/jira/browse/HBASE-18183 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[
https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174755#comment-16174755
]
Abhishek Singh Chouhan commented on HBASE-18796:
Spent some time looking at the failure. Looks to be a problem elsewhere that
surfaced.
The test does a split and then tries a batch get operation which fails due to
table not found although the table is there. This is happening because now that
we do not put daughter locations before they're actually opened on the
regionserver, we run into NoServerForRegionException in
ConnectionImplementation#locateRegionInMeta which should be fine since there
are retries which should succeed as soon as the region is opened. However our
retry fails on a TableNotFound exception here
{code}
try (ReversedClientScanner rcs =
new ReversedClientScanner(conf, s, TableName.META_TABLE_NAME, this,
rpcCallerFactory,
rpcControllerFactory, getMetaLookupPool(),
metaReplicaCallTimeoutScanInMicroSecond)) {
regionInfoRow = rcs.next();
}
if (regionInfoRow == null) {
throw new TableNotFoundException(tableName);
}
{code}
The result that we get has mayHaveMoreCellsInRow() true during one of the
retries, since we don't have setAllowPartialResults(true) set on our scan we
get regionInfoRow as null since we got only 1 row which has
mayHaveMoreCellsInRow() as true and we use
CompleteScanResultCache which won't return this to the client. After i do
{code}
s.addFamily(HConstants.CATALOG_FAMILY);
s.setOneRowLimit();
+ s.setAllowPartialResults(true);
if (this.useMetaReplicas) {
s.setConsistency(Consistency.TIMELINE);
}
{code}
the client is able to ride over the split during its retries and the test
passes.
[~tedyu] [~apurtell] This issues seems to be something that can be hit during
any other retry too in locateRegionInMeta when mayHaveMoreCellsInRow() is true
for the meta scan and the client would get TableNotFound and will not retry. I
can open another jira for this if this sounds good.
> Admin#isTableAvailable returns incorrect result before daughter regions are
> opened
> --
>
> Key: HBASE-18796
> URL: https://issues.apache.org/jira/browse/HBASE-18796
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.1
>Reporter: Abhishek Singh Chouhan
>Assignee: Abhishek Singh Chouhan
> Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0
>
> Attachments: HBASE-18796.branch-1.001.patch,
> HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch,
> HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch
>
>
> Admin#isTableAvailable checks if it can getServerName for the meta entries it
> reads. During the time of split server location are added to the meta entries
> in MetaTableAccessor#splitRegion although the description of the method says
> "Does not add the location information to the daughter regions since they are
> not open yet.". At this point during the split daughter regions are not
> actually open, so we can get to a state where parent is offline, daughters
> are not yet open but isTableAvailable returns true.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Summary: [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private (was: [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174948#comment-16174948 ] Ted Yu commented on HBASE-18796: Since the issue was found soon after the commit, I would think applying addendum should suffice. Thanks for digging. > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-14845) hbase-server leaks jdk.tools dependency to mapreduce consumers
[
https://issues.apache.org/jira/browse/HBASE-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175048#comment-16175048
]
Sean Busbey commented on HBASE-14845:
-
this got fixed in HBase 2+ via the breakout into a mapreduce module:
{code}
[INFO]
[INFO] Building Apache HBase - MapReduce 2.0.0-alpha3-SNAPSHOT
[INFO]
[WARNING] The POM for
org.apache.hbase:hbase-server:jar:2.0.0-alpha3-20170913.213312-2 is invalid,
transitive dependencies (if any) will not be available, enable debug logging
for more details
[WARNING] The POM for
org.apache.hbase:hbase-server:jar:tests:2.0.0-alpha3-20170913.213312-2 is
invalid, transitive dependencies (if any) will not be available, enable debug
logging for more details
[INFO]
[INFO] --- maven-dependency-plugin:3.0.1:tree (default-cli) @ hbase-mapreduce
---
[INFO] org.apache.hbase:hbase-mapreduce:jar:2.0.0-alpha3-SNAPSHOT
[INFO] +-
org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:1.0.1:compile
[INFO] +- org.apache.hbase.thirdparty:hbase-shaded-netty:jar:1.0.1:compile
[INFO] +- org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:1.0.1:compile
[INFO] +- org.apache.hbase:hbase-common:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] | +- commons-codec:commons-codec:jar:1.10:compile
[INFO] | +- org.apache.commons:commons-collections4:jar:4.1:compile
[INFO] | \- org.apache.commons:commons-crypto:jar:1.0.0:compile
[INFO] +- org.apache.hbase:hbase-protocol:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] +-
org.apache.hbase:hbase-protocol-shaded:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-metrics:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-metrics-api:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- io.dropwizard.metrics:metrics-core:jar:3.2.1:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.24:compile
[INFO] +- org.apache.hbase:hbase-prefix-tree:jar:2.0.0-alpha3-SNAPSHOT:runtime
[INFO] +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:compile
[INFO] +- org.apache.hbase:hbase-client:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] | +- org.jruby.jcodings:jcodings:jar:1.0.18:compile
[INFO] | +- org.jruby.joni:joni:jar:2.1.11:compile
[INFO] | +- org.apache.curator:curator-framework:jar:2.12.0:compile
[INFO] | +- org.apache.curator:curator-client:jar:2.12.0:compile
[INFO] | \- org.apache.hadoop:hadoop-auth:jar:2.7.1:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
[INFO] | \-
org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] |+-
org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] |+- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] |\- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] +- org.apache.hbase:hbase-hadoop-compat:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +-
org.apache.hbase:hbase-hadoop2-compat:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-server:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-replication:jar:2.0.0-alpha3-SNAPSHOT:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] +- commons-cli:commons-cli:jar:1.4:compile
[INFO] +- commons-io:commons-io:jar:2.5:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.6:compile
[INFO] +- commons-logging:commons-logging:jar:1.2:compile
[INFO] +- org.apache.zookeeper:zookeeper:jar:3.4.10:compile
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] +- com.github.stephenc.findbugs:findbugs-annotations:jar:1.3.9-1:compile
(optional)
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | +- org.apache.commons:commons-math3:jar:3.1.1:compile
[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] | +- commons-net:commons-net:jar:3.1:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] | +- org.apache.curator:curator-recipes:jar:2.12.0:compile
[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile
[INFO] | \- org.tukaani:xz:jar:1.0:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile
[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1:compile
[INFO] | \-
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[
https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175010#comment-16175010
]
Hadoop QA commented on HBASE-18731:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
58s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
32s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18731 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888313/HBASE-18731.0.patch |
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7ac0b8b2e269 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a6c3c64 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8723/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8723/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message
[jira] [Comment Edited] (HBASE-18731) [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed
[
https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157740#comment-16157740
]
Sean Busbey edited comment on HBASE-18731 at 9/21/17 2:45 PM:
--
presumably this can also cover
{code}public static SetQuotaRequest buildSetQuotaRequestProto(final
QuotaSettings settings){code} in QuotaSettings? Or is there a different jira
for that already?
QuotaSettings is an Admin API and this is an instance of protobufs in our
public API. Better maybe to just release note it and make sure only a POJO is
within the public API for 2.0?
was (Author: busbey):
presumably this can also cover
{code}public static SetQuotaRequest buildSetQuotaRequestProto(final
QuotaSettings settings){code} in QuotaSettings? Or is there a different jira
for that already?
QuotaSettings is an Admin API and this is an instance of protobufs in our
public API. Better maybe to just release not it and make sure only a POJO is
within the public API for 2.0?
> [compat 1-2] QuotaSettings#setupSetQuotaRequest param changed
> -
>
> Key: HBASE-18731
> URL: https://issues.apache.org/jira/browse/HBASE-18731
> Project: HBase
> Issue Type: Sub-task
> Components: API
>Reporter: stack
>Assignee: Sean Busbey
> Fix For: 2.0.0-alpha-4
>
>
> QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to
> package
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest.
> QuotaSettings was added in 1.1.
> Is QuotaSettings for use by anyone but our CPEP?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Attachment: HBASE-18731.0.patch -0 master /branch-2 - mark the relevant methods as IA.Private for branch-1 and earlier, I'll add a Deprecated marker with a warning about the removal in 2+ > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174998#comment-16174998 ] Ted Yu commented on HBASE-18651: Planning to commit later today if there is no further review comment. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18651) Let ChaosMonkeyRunner expose the chaos monkey runner it creates
[ https://issues.apache.org/jira/browse/HBASE-18651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175001#comment-16175001 ] Reid Chan commented on HBASE-18651: --- ping [~mdrob], would you mind taking a look. > Let ChaosMonkeyRunner expose the chaos monkey runner it creates > --- > > Key: HBASE-18651 > URL: https://issues.apache.org/jira/browse/HBASE-18651 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Reid Chan > Attachments: HBASE-18651.master.001.patch, > HBASE-18651.master.002.patch, HBASE-18651.master.003.patch, > HBASE-18651.master.004.patch, HBASE-18651.master.005.patch, > HBASE-18651.master.006.patch > > > Currently ChaosMonkeyRunner#main() instantiates ChaosMonkeyRunner without > keeping track of the instance. > This poses some challenge when ChaosMonkeyRunner is used programmatically > because the caller cannot get hold of the runner. > As [~mdrob] suggested, we should expose the chaos monkey runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18857) Update News section to point to video / slides from prior conferences
Sean Busbey created HBASE-18857: --- Summary: Update News section to point to video / slides from prior conferences Key: HBASE-18857 URL: https://issues.apache.org/jira/browse/HBASE-18857 Project: HBase Issue Type: Improvement Components: website Reporter: Sean Busbey or maybe we need a history of "talks / meetups" section. In either case, right now the new section is the closest we have to pointers on community provided resources. Adding in videos / slides would be a great addition to hte CFP pages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-18731: Status: Patch Available (was: Open) > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174951#comment-16174951 ] ramkrishna.s.vasudevan commented on HBASE-16769: Comment on withCpCall already been said by [~chia7712]. That needs to be fixed. The CP hook is changing now. So since we are changing the signatures already it is fine. So we should revisit all CP hooks if there are any other similar hooks with unwanted hooks that is passing some private things and some unnecessary/additional things as param. Why is that we are newly passing the new param 'withCphook' and the other deleteSnapshot() API do not have that switch? Rest looks good to me. bq.AC should be internal, not as a CP? Good idea but probably a big change. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18856) take find security bugs for a spin
Sean Busbey created HBASE-18856: --- Summary: take find security bugs for a spin Key: HBASE-18856 URL: https://issues.apache.org/jira/browse/HBASE-18856 Project: HBase Issue Type: Improvement Components: security, test Reporter: Sean Busbey Priority: Minor it'd be nice to see if the [Find Security Bugs|http://find-sec-bugs.github.io/] plugin for FindBugs works now that we've switched to SpotBugs. Presuming it does, getting an initial include/exclude list going and things merged into our build could be a good addition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Component/s: documentation > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175043#comment-16175043 ] stack commented on HBASE-17730: --- Yes is answer to your question above [~anoop.hbase] -- smile. I have list of incompatibilities for coprocessors running here https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.9k7mjbauv0wj but this issue is about more than just list of breakage; it is about adding section to 2.0 migration on how to move coprocessors over. > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17730) Migration to 2.0 for coprocessors
[ https://issues.apache.org/jira/browse/HBASE-17730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17730: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0 > Migration to 2.0 for coprocessors > -- > > Key: HBASE-17730 > URL: https://issues.apache.org/jira/browse/HBASE-17730 > Project: HBase > Issue Type: Sub-task > Components: documentation, migration >Reporter: Appy >Priority: Blocker > Fix For: 2.0.0 > > > Jiras breaking coprocessor compatibility should be marked with component ' > Coprocessor', and label 'incompatible'. > Close to releasing 2.0, we should go through all such jiras and write down > steps for migrating coprocessor easily. > The idea is, it might be very hard to fix coprocessor breakages by reverse > engineering errors, but will be easier we suggest easiest way to fix > breakages resulting from each individual incompatible change. > For eg. HBASE-17312 is incompatible change. It'll result in 100s of errors > because BaseXXXObserver classes are gone and will probably result in a lot of > confusion, but if we explicitly mention the fix which is just one line change > - replace "Foo extends BaseXXXObserver" with "Foo implements XXXObserver" - > it makes it very easy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174783#comment-16174783
]
Hadoop QA commented on HBASE-13346:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red}499m
54s{color} | {color:red} Docker failed to build yetus/hbase:5d60123. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-13346 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888211/HBASE-13346.master.003.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8722/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18823) Apply RegionInfo to MasterObserver/RegionObserver/WALObserver
[ https://issues.apache.org/jira/browse/HBASE-18823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174328#comment-16174328 ] Hudson commented on HBASE-18823: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3751 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3751/]) HBASE-18823 Apply RegionInfo to (appy: rev a6c3c645fd193f1805d449d91528e2a9246f8ff8) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithAbort.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterCoprocessorExceptionWithRemove.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestRegionReplicaReplicationEndpointNoMaster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/BaseTestHBaseFsck.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * (edit) hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverForAddingMutationsFromCoprocessors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/namespace/TestNamespaceAuditor.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorMetrics.java * (edit) hbase-examples/src/main/java/org/apache/hadoop/hbase/coprocessor/example/ExampleMasterObserverWithMetrics.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionCoprocessorEnvironment.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/WALObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALObserver.java * (edit) hbase-rsgroup/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupAdminEndpoint.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/CoprocessorWhitelistMasterObserver.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestEnableTable.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java > Apply RegionInfo to MasterObserver/RegionObserver/WALObserver > - > > Key: HBASE-18823 > URL: https://issues.apache.org/jira/browse/HBASE-18823 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Labels: incompatible > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18823.v0.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174335#comment-16174335 ] Chia-Ping Tsai commented on HBASE-18010: It seems the failed tests are related to the patch. [~anastas] WDYT? > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Fix For: 3.0.0 > > Attachments: HBASE-18010-branch-2.patch, HBASE-18010-V04.patch, > HBASE-18010-V06.patch, HBASE-18010-V07.patch, HBASE-18010-V08.patch, > HBASE-18010-V09.patch, HBASE-18010-V10.patch, HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18160) Fix incorrect logic in FilterList.filterKeyValue
[
https://issues.apache.org/jira/browse/HBASE-18160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174350#comment-16174350
]
Hadoop QA commented on HBASE-18160:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
56s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
36m 19s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
29s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}155m 11s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}217m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:5d60123 |
| JIRA Issue | HBASE-18160 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12888194/HBASE-18160.v7.patch |
| Optional Tests | asflicense shadedjars javac javadoc unit findbugs
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f3d8d3559cb9 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HBASE-18796) Admin#isTableAvailable returns incorrect result before daughter regions are opened
[ https://issues.apache.org/jira/browse/HBASE-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174389#comment-16174389 ] Abhishek Singh Chouhan commented on HBASE-18796: Looking into it. Thanks [~tedyu]. > Admin#isTableAvailable returns incorrect result before daughter regions are > opened > -- > > Key: HBASE-18796 > URL: https://issues.apache.org/jira/browse/HBASE-18796 > Project: HBase > Issue Type: Bug >Affects Versions: 1.3.1 >Reporter: Abhishek Singh Chouhan >Assignee: Abhishek Singh Chouhan > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0 > > Attachments: HBASE-18796.branch-1.001.patch, > HBASE-18796.branch-1.001.patch, HBASE-18796.branch-1.002.patch, > HBASE-18796.branch-1.003.patch, HBASE-18796.master.001.patch > > > Admin#isTableAvailable checks if it can getServerName for the meta entries it > reads. During the time of split server location are added to the meta entries > in MetaTableAccessor#splitRegion although the description of the method says > "Does not add the location information to the daughter regions since they are > not open yet.". At this point during the split daughter regions are not > actually open, so we can get to a state where parent is offline, daughters > are not yet open but isTableAvailable returns true. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-17782:
--
Description: As per discussed in HBASE-17747, we need to make it
configurable to decide whether to use weak or soft reference for
{{IdReadWriteLock}}, with soft reference by default. (was: # As per discussed
in HBASE-17747, we need to make it configurable to decide whether to use weak
or soft reference for {{IdReadWriteLock}}, with soft reference by default.)
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-17782) Extend IdReadWriteLock to support using both weak and soft reference
[
https://issues.apache.org/jira/browse/HBASE-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174325#comment-16174325
]
Yu Li commented on HBASE-17782:
---
[~chenyechao] I guess you changed the description by accident and have revert
it, please let me know if any concern here. Thanks.
> Extend IdReadWriteLock to support using both weak and soft reference
>
>
> Key: HBASE-17782
> URL: https://issues.apache.org/jira/browse/HBASE-17782
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-17782.patch, HBASE-17782.patch
>
>
> As per discussed in HBASE-17747, we need to make it configurable to decide
> whether to use weak or soft reference for {{IdReadWriteLock}}, with soft
> reference by default.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18731) [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf internals as IA.Private
[ https://issues.apache.org/jira/browse/HBASE-18731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175084#comment-16175084 ] stack commented on HBASE-18731: --- +1 on patch for branch-2 > [compat 1-2] Mark protected methods of QuotaSettings that touch Protobuf > internals as IA.Private > > > Key: HBASE-18731 > URL: https://issues.apache.org/jira/browse/HBASE-18731 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: stack >Assignee: Sean Busbey > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18731.0.patch > > > QuotaSettings is Audience public. The param for setupSetQuotaRequest moved to > package > org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.SetQuotaRequest. > QuotaSettings was added in 1.1. > Is QuotaSettings for use by anyone but our CPEP? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
Chia-Ping Tsai created HBASE-18858:
--
Summary: Avoid NPE occurring in
TestReplicationBase#tearDownAfterClass
Key: HBASE-18858
URL: https://issues.apache.org/jira/browse/HBASE-18858
Project: HBase
Issue Type: Bug
Components: test
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Minor
Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
{code}
public static void tearDownAfterClass() throws Exception {
htable2.close();
htable1.close();
admin.close();
utility2.shutdownMiniCluster();
utility1.shutdownMiniCluster();
}
{code}
If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16769) Deprecate/remove PB references from MasterObserver and RegionServerObserver
[ https://issues.apache.org/jira/browse/HBASE-16769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175264#comment-16175264 ] stack commented on HBASE-16769: --- Sounds good to me [~anoop.hbase] Put up patch so I can +1 it. Add that the internal use is by AC for us devs reading along afterward. > Deprecate/remove PB references from MasterObserver and RegionServerObserver > --- > > Key: HBASE-16769 > URL: https://issues.apache.org/jira/browse/HBASE-16769 > Project: HBase > Issue Type: Bug >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-16769.patch, HBASE-16769_V2.patch > > > This is effectively a sub-task for HBASE-15174. > CP Methods > MasterObserver > preListSnapshot > postListSnapshot > preSnapshot > postSnapshot > preCloneSnapshot > postCloneSnapshot > preRestoreSnapshot > postRestoreSnapshot > preDeleteSnapshot > postDeleteSnapshot > > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetUserQuota > postSetUserQuota > preSetTableQuota > postSetTableQuota > preSetNamespaceQuota > postSetNamespaceQuota > > RegionServerObserver > preReplicateLogEntries > postReplicateLogEntries > Note : This issue not handling Quota related CPs. Same is handled by a > subtask here HBase-18807 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175269#comment-16175269
]
stack commented on HBASE-13346:
---
I just read HBASE-18797 My review was done BEFORE I read HBASE-18797 (In other
words, I don't think it very useful).
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18838: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
[
https://issues.apache.org/jira/browse/HBASE-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18858:
---
Status: Patch Available (was: Open)
> Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
> -
>
> Key: HBASE-18858
> URL: https://issues.apache.org/jira/browse/HBASE-18858
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
>
> Attachments: HBASE-18858.branch-1.v0.patch
>
>
> {code}
> public static void tearDownAfterClass() throws Exception {
> htable2.close();
> htable1.close();
> admin.close();
> utility2.shutdownMiniCluster();
> utility1.shutdownMiniCluster();
> }
> {code}
> If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
> all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18858) Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
[
https://issues.apache.org/jira/browse/HBASE-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18858:
---
Attachment: HBASE-18858.branch-1.v0.patch
> Avoid NPE occurring in TestReplicationBase#tearDownAfterClass
> -
>
> Key: HBASE-18858
> URL: https://issues.apache.org/jira/browse/HBASE-18858
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 1.4.0, 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-1
>
> Attachments: HBASE-18858.branch-1.v0.patch
>
>
> {code}
> public static void tearDownAfterClass() throws Exception {
> htable2.close();
> htable1.close();
> admin.close();
> utility2.shutdownMiniCluster();
> utility1.shutdownMiniCluster();
> }
> {code}
> If the setUpBeforeClass() fail, some members will be NULL. We need to ensures
> all resources are closed at the end of tests.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-17852: -- Fix Version/s: (was: 2.0.0-alpha-4) 2.0.0-beta-1 > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work stopped] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18838 stopped by Sean Busbey. --- > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17852) Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175272#comment-16175272 ] stack commented on HBASE-17852: --- Moving out of alpha-4. If it comes in before alpha-4, that'd be great but no critical to the coprocessor-focused release. > Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental > backup) > > > Key: HBASE-17852 > URL: https://issues.apache.org/jira/browse/HBASE-17852 > Project: HBase > Issue Type: Sub-task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-17852-v1.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18838) shaded artifacts are incorrect when built against hadoop 3
[ https://issues.apache.org/jira/browse/HBASE-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reassigned HBASE-18838: --- Assignee: (was: Sean Busbey) I'll be happy to pick this back up when I start looking at beta-1 stuff, but I'm setting it back to unassigned at the moment in case someone else wants to dig into it earlier. > shaded artifacts are incorrect when built against hadoop 3 > -- > > Key: HBASE-18838 > URL: https://issues.apache.org/jira/browse/HBASE-18838 > Project: HBase > Issue Type: Bug > Components: Client >Affects Versions: 2.0.0-alpha-3 >Reporter: Sean Busbey >Priority: Critical > Fix For: 2.0.0-beta-1 > > > Building master/branch-2 against the hadoop-3 profile results in > check-invariants screaming about unrelocated dependencies. will list details > in comment. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
