[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558885#comment-16558885 ] Alex Leblang commented on HBASE-19369: -- [~mdrob] I stalled on this patch and then unfortunately forgot about it. It is now all yours > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang >Priority: Major > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch, HBASE-19369.v5.patch, HBASE-19369.v6.patch, > HBASE-19369.v7.patch, HBASE-19369.v8.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298877#comment-16298877 ] Alex Leblang commented on HBASE-19369: -- I'll look into that. I think that we probably can > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298859#comment-16298859 ] Alex Leblang commented on HBASE-19369: -- The problem is that the version of org.apache.hadoop.hdfs.DFSTestUtil that we are bringing in doesn't have the following method https://github.com/apache/hadoop/blob/branch-3.0.0/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java#L297 I'm looking into why that is > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-19369: - Attachment: HBASE-19369.master.004.patch > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch, > HBASE-19369.master.004.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287696#comment-16287696 ] Alex Leblang commented on HBASE-19369: -- I had an issue where I mixed up the wrong version of ProtobufLogWriter in this patch. I'm working on fixing that > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-19369: - Attachment: HBASE-19369.master.003.patch > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch, HBASE-19369.master.003.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-19369: - Attachment: HBASE-19369.master.002.patch > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch, > HBASE-19369.master.002.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272804#comment-16272804 ] Alex Leblang commented on HBASE-19369: -- I should have added a test to go along with this. I will do that. I will also add a code comment explaining that with erasure coding we should use the builder pattern to avoid replicating from a non-ec to an ec directory > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-19369 started by Alex Leblang. > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-19369: - Status: Patch Available (was: In Progress) > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-19369: - Attachment: HBASE-19369.master.001.patch > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-19369.master.001.patch > > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
[ https://issues.apache.org/jira/browse/HBASE-19369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270037#comment-16270037 ] Alex Leblang commented on HBASE-19369: -- Yes, I will try to post a patch tomorrow > HBase Should use Builder Pattern to Create Log Files while using WAL on > Erasure Coding > -- > > Key: HBASE-19369 > URL: https://issues.apache.org/jira/browse/HBASE-19369 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Alex Leblang >Assignee: Alex Leblang > > Right now an HBase instance using the WAL won't function properly in an > Erasure Coded environment. We should change the following line to use the > hdfs.DistributedFileSystem builder pattern > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19369) HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding
Alex Leblang created HBASE-19369: Summary: HBase Should use Builder Pattern to Create Log Files while using WAL on Erasure Coding Key: HBASE-19369 URL: https://issues.apache.org/jira/browse/HBASE-19369 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Alex Leblang Assignee: Alex Leblang Right now an HBase instance using the WAL won't function properly in an Erasure Coded environment. We should change the following line to use the hdfs.DistributedFileSystem builder pattern https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L92 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020-addendum.001.patch This patch doesn't contain the whole commit, just the fix for the problem that Andrew found > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020-addendum.001.patch, > HBASE-18020.branch-1.2.001.patch, HBASE-18020.branch-1.2.002.patch, > HBASE-18020.branch-1.2.003.patch, HBASE-18020.branch-1.2.004.patch, > HBASE-18020.branch-1.2.005.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.005.patch For some reason, perhaps due to the change from bash to python, the code needed to explicitly state "origin/branch_name" on clean checkouts. I went back to look at the bash version and we use the same command, git rev-parse, so I'm not sure why this only just starting failing but this fixed the issue in all cases where I was able to replicate the failure > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch, HBASE-18020.branch-1.2.005.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121797#comment-16121797 ] Alex Leblang commented on HBASE-18020: -- I was away from my computer yesterday, but I am looking into this now [~andrew.purt...@gmail.com] [~busbey] > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.12 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100418#comment-16100418 ] Alex Leblang commented on HBASE-18020: -- Thanks [~dimaspivak]! > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095406#comment-16095406 ] Alex Leblang commented on HBASE-18406: -- Ok, the latest patch keeps that logging line > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18406.master.001.patch, > HBASE-18406.master.002.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18406: - Attachment: HBASE-18406.master.002.patch > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18406.master.001.patch, > HBASE-18406.master.002.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095373#comment-16095373 ] Alex Leblang commented on HBASE-18020: -- ping [~dimaspivak], would you mind taking a look to ensure that I've properly responded to your review? > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18417) Add tests for scripts under bin/
[ https://issues.apache.org/jira/browse/HBASE-18417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093702#comment-16093702 ] Alex Leblang commented on HBASE-18417: -- I'll be happy to review. I'm still new to the HBase apache jenkins infra story so it'll be a good way to see what's going on. > Add tests for scripts under bin/ > > > Key: HBASE-18417 > URL: https://issues.apache.org/jira/browse/HBASE-18417 > Project: HBase > Issue Type: Improvement > Components: scripts, test >Reporter: Mike Drob > Fix For: 2.0.0 > > > Currently, we have no way to test that the scripts under bin/ work and we > only see problems when somebody tries to use them and reports an issue. We > should figure out some automated testing for them so we know when they break > at commit time. > See also: HBASE-18393 HBASE-17988 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18417) Add tests for scripts under bin/
[ https://issues.apache.org/jira/browse/HBASE-18417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093672#comment-16093672 ] Alex Leblang commented on HBASE-18417: -- I think that that method you describe [~busbey] seems like a good option. I'd definitely prefer to avoid trying to add a lot of non-java script execution from maven > Add tests for scripts under bin/ > > > Key: HBASE-18417 > URL: https://issues.apache.org/jira/browse/HBASE-18417 > Project: HBase > Issue Type: Improvement > Components: scripts, test >Reporter: Mike Drob > Fix For: 2.0.0 > > > Currently, we have no way to test that the scripts under bin/ work and we > only see problems when somebody tries to use them and reports an issue. We > should figure out some automated testing for them so we know when they break > at commit time. > See also: HBASE-18393 HBASE-17988 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
[ https://issues.apache.org/jira/browse/HBASE-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18406: - Attachment: HBASE-18406.master.001.patch > In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op > - > > Key: HBASE-18406 > URL: https://issues.apache.org/jira/browse/HBASE-18406 > Project: HBase > Issue Type: Bug >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18406.master.001.patch > > > The comments above this method explain that it exists to set configs and > return, however, no configs are set in the method. > As you can see here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 > > It is only ever called here: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18406) In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op
Alex Leblang created HBASE-18406: Summary: In ServerCrashProcedure.java start(MasterProcedureEnv) is a no-op Key: HBASE-18406 URL: https://issues.apache.org/jira/browse/HBASE-18406 Project: HBase Issue Type: Bug Reporter: Alex Leblang The comments above this method explain that it exists to set configs and return, however, no configs are set in the method. As you can see here: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L210-L214 It is only ever called here: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ServerCrashProcedure.java#L142 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16092137#comment-16092137 ] Alex Leblang commented on HBASE-18147: -- lgtm, I didn't see any issues > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088212#comment-16088212 ] Alex Leblang commented on HBASE-18020: -- I think I've addressed all of your concerns [~dspivak] > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.004.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch, > HBASE-18020.branch-1.2.004.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087756#comment-16087756 ] Alex Leblang commented on HBASE-18147: -- I think that given the jdk change this is a good move > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey >Assignee: Sean Busbey > Attachments: HBASE-18147.0.patch, HBASE-18147-branch-1.v1.patch, > HBASE-18147.v1.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078604#comment-16078604 ] Alex Leblang edited comment on HBASE-18020 at 7/7/17 8:04 PM: -- I posted an updated version to rb. Thanks for the thorough review [~dspivak] was (Author: awleblang): I posted an updated version to rb. Thanks for the through review [~dspivak] > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078604#comment-16078604 ] Alex Leblang commented on HBASE-18020: -- I posted an updated version to rb. Thanks for the through review [~dspivak] > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.003.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch, HBASE-18020.branch-1.2.003.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16067055#comment-16067055 ] Alex Leblang commented on HBASE-18020: -- Very excited to see the pylint file in action! I think this should resolve the issue but we'll see > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.002.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch, > HBASE-18020.branch-1.2.002.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16066935#comment-16066935 ] Alex Leblang commented on HBASE-18041: -- Woohoo! Thanks Sean! > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.2.7, 2.0.0-alpha-2, 1.1.12 > > Attachments: diff-patch-pylint-noRC.txt, diff-patch-pylint.txt, > HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065492#comment-16065492 ] Alex Leblang commented on HBASE-18147: -- That patch looks good to me [~busbey] I'll give it a + > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey > Attachments: HBASE-18147.0.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064006#comment-16064006 ] Alex Leblang commented on HBASE-18020: -- I added a new version. Hopefully we can get this in. Here's a link to the reviewboard https://reviews.apache.org/r/60449/ > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.branch-1.2.001.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch, HBASE-18020.branch-1.2.001.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18041: - Attachment: diff-patch-pylint.txt diff-patch-pylint-noRC.txt Here is a run with the pylint file. There are 790 issues with the file. I've also attached a run without the RC file. Without a pylintrc file there are 15600 issues > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: diff-patch-pylint-noRC.txt, diff-patch-pylint.txt, > HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18147) nightly job to check health of active branches
[ https://issues.apache.org/jira/browse/HBASE-18147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039746#comment-16039746 ] Alex Leblang commented on HBASE-18147: -- I had heard that yetus was end of lifeing versions 0.1.0 - 0.3.0 (see here https://issues.apache.org/jira/browse/YETUS-520 ) so perhaps we should look into upgrading to 0.4 > nightly job to check health of active branches > -- > > Key: HBASE-18147 > URL: https://issues.apache.org/jira/browse/HBASE-18147 > Project: HBase > Issue Type: Test > Components: community, test >Reporter: Sean Busbey > Attachments: HBASE-18147.0.patch > > > We should set up a job that runs Apache Yetus Test Patch's nightly mode. > Essentially, it produces a report that considers how the branch measures up > against the things we check in our precommit checks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18148) automated tests against non-minicluster based deployment
[ https://issues.apache.org/jira/browse/HBASE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033881#comment-16033881 ] Alex Leblang commented on HBASE-18148: -- Have you tested this recently? I filed this clusterdock issue b/c I was unable to build the images https://github.com/clusterdock/topology_apache_hbase/issues/6 > automated tests against non-minicluster based deployment > > > Key: HBASE-18148 > URL: https://issues.apache.org/jira/browse/HBASE-18148 > Project: HBase > Issue Type: Test > Components: community, integration tests >Reporter: Sean Busbey > Attachments: HBASE-18148.0.patch > > > we should have ITs that run automatically (i.e. nightly) against a cluster > that isn't a minicluster or standalone instance. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18148) automated tests against non-minicluster based deployment
[ https://issues.apache.org/jira/browse/HBASE-18148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033416#comment-16033416 ] Alex Leblang commented on HBASE-18148: -- Could you include the default values for the parameters in the script? This would make it easier for reviewers without jenkins access to follow the script. > automated tests against non-minicluster based deployment > > > Key: HBASE-18148 > URL: https://issues.apache.org/jira/browse/HBASE-18148 > Project: HBase > Issue Type: Test > Components: community, integration tests >Reporter: Sean Busbey > Attachments: HBASE-18148.0.patch > > > we should have ITs that run automatically (i.e. nightly) against a cluster > that isn't a minicluster or standalone instance. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020354#comment-16020354 ] Alex Leblang commented on HBASE-18041: -- [~dspivak] what do we need to do to move this forward? Do you want me to go through the checks I proposed not to use one by one? Should we try to get more opinions? > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009048#comment-16009048 ] Alex Leblang commented on HBASE-18041: -- Well I just took the yetus pylintrc file so I can't be sure, but I know that linters can be pretty strict. We could start from another spot, but the things disabled weren't had selected to make our current codebase pass. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009048#comment-16009048 ] Alex Leblang edited comment on HBASE-18041 at 5/13/17 2:13 AM: --- Well I just took the yetus pylintrc file, but I know that linters can be pretty strict. We could start from another spot. The things disabled weren't hand selected to make our current codebase pass. was (Author: awleblang): Well I just took the yetus pylintrc file so I can't be sure, but I know that linters can be pretty strict. We could start from another spot, but the things disabled weren't had selected to make our current codebase pass. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18041: - Status: Patch Available (was: In Progress) > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Work started] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18041 started by Alex Leblang. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18041: - Attachment: HBASE-18041.branch-1.2.001.patch > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang >Assignee: Alex Leblang > Attachments: HBASE-18041.branch-1.2.001.patch > > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008619#comment-16008619 ] Alex Leblang commented on HBASE-18041: -- Sure, I'll make a first attempt at this. > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement > Components: community >Reporter: Alex Leblang > > Yetus runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetus defaults. > As an argument for this, the yetus project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18041) Add pylintrc file to HBase
[ https://issues.apache.org/jira/browse/HBASE-18041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16008467#comment-16008467 ] Alex Leblang commented on HBASE-18041: -- This was prompted by some discussion in https://issues.apache.org/jira/browse/HBASE-18020. One issue that came up there was 2 versus 4 space indents in python. HBase uses both currently, though there aren't that many python files. Clusterdock uses 4 spaces, hadoop uses 2 spaces, and yetis uses two spaces. [~dimaspivak] expressed a desire to use 4 spaces > Add pylintrc file to HBase > -- > > Key: HBASE-18041 > URL: https://issues.apache.org/jira/browse/HBASE-18041 > Project: HBase > Issue Type: Improvement >Reporter: Alex Leblang > > Yetis runs all commits with python files through a linter. I think that the > HBase community should add a pylintrc file to actively choose the project's > python style instead of just relying on yetis defaults. > As an argument for this, the yetis project itself doesn't even use the > default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18041) Add pylintrc file to HBase
Alex Leblang created HBASE-18041: Summary: Add pylintrc file to HBase Key: HBASE-18041 URL: https://issues.apache.org/jira/browse/HBASE-18041 Project: HBase Issue Type: Improvement Reporter: Alex Leblang Yetis runs all commits with python files through a linter. I think that the HBase community should add a pylintrc file to actively choose the project's python style instead of just relying on yetis defaults. As an argument for this, the yetis project itself doesn't even use the default python linter for its own commits. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007302#comment-16007302 ] Alex Leblang commented on HBASE-18020: -- As for the python question, I'd prefer to keep it in python 2. The patch is already in python 2 and we use python 2 across hbase so committing another python 2 file seems noncontroversial. Python 2 and 3 aren't always very compatible and, as you say, python 3 has been our for 10 years yet python 2 remains more popular. I see no need to make a change that potentially worsens compatibility and brings the project to a version of a language that is less well known and used. > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007286#comment-16007286 ] Alex Leblang commented on HBASE-18020: -- If I'm not mistaken, I believe that we currently do not have an hbase project pylint rc file. We should make one! That would presumably require us to decide on some stylistic questions. I kept the spacing here at 2 spaces as that's how it was in hadoop. I couldn't find a uniform standard in the hbase code base. Some files go with two spaces and other use four. In fact, this file seems to use both (which is ironic considering its name) https://github.com/apache/hbase/blob/master/dev-support/checkstyle_report.py > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Attachment: HBASE-18020.0.patch > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
[ https://issues.apache.org/jira/browse/HBASE-18020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Leblang updated HBASE-18020: - Status: Patch Available (was: Open) > Update API Compliance Checker to Incorporate Improvements Done in Hadoop > > > Key: HBASE-18020 > URL: https://issues.apache.org/jira/browse/HBASE-18020 > Project: HBase > Issue Type: Improvement > Components: API, community >Reporter: Alex Leblang >Assignee: Alex Leblang > Fix For: 2.0.0 > > Attachments: HBASE-18020.0.patch > > > Recently the Hadoop community has made a number of improvements in their api > compliance checker based on feedback from the hbase and kudu community. We > should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HBASE-18020) Update API Compliance Checker to Incorporate Improvements Done in Hadoop
Alex Leblang created HBASE-18020: Summary: Update API Compliance Checker to Incorporate Improvements Done in Hadoop Key: HBASE-18020 URL: https://issues.apache.org/jira/browse/HBASE-18020 Project: HBase Issue Type: Improvement Components: API, community Reporter: Alex Leblang Recently the Hadoop community has made a number of improvements in their api compliance checker based on feedback from the hbase and kudu community. We should adopt these changes ourselves. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16441) Document use of Apache HBase clusterdock topology
[ https://issues.apache.org/jira/browse/HBASE-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940840#comment-15940840 ] Alex Leblang commented on HBASE-16441: -- Well in my ideal world I would have been a consumer. But maybe I will after I get some builds running > Document use of Apache HBase clusterdock topology > - > > Key: HBASE-16441 > URL: https://issues.apache.org/jira/browse/HBASE-16441 > Project: HBase > Issue Type: New Feature > Components: documentation >Reporter: Dima Spivak >Assignee: Dima Spivak > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-16441) Document use of Apache HBase clusterdock topology
[ https://issues.apache.org/jira/browse/HBASE-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940807#comment-15940807 ] Alex Leblang commented on HBASE-16441: -- I'm assuming that this work was never started? > Document use of Apache HBase clusterdock topology > - > > Key: HBASE-16441 > URL: https://issues.apache.org/jira/browse/HBASE-16441 > Project: HBase > Issue Type: New Feature > Components: documentation >Reporter: Dima Spivak >Assignee: Dima Spivak > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HBASE-12852) Tests from hbase-it that use ChaosMonkey don't fail if SSH commands fail
[
https://issues.apache.org/jira/browse/HBASE-12852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816830#comment-15816830
]
Alex Leblang commented on HBASE-12852:
--
I'm still dealing with this issue, so I agree that it would be nice to have
> Tests from hbase-it that use ChaosMonkey don't fail if SSH commands fail
>
>
> Key: HBASE-12852
> URL: https://issues.apache.org/jira/browse/HBASE-12852
> Project: HBase
> Issue Type: Bug
> Components: integration tests
>Affects Versions: 0.98.6
>Reporter: Dima Spivak
>Assignee: Dima Spivak
>
> I've just started rolling my sleeves up and playing about with hbase-it (at
> the moment, only on 0.98.6), but wanted to begin filing JIRAs for issues I
> encounter so that I don't forget to get to them. First up is the fact that it
> seems that tests run with ChaosMonkey don't fail when the ChaosMonkey fails
> to work. As an example, while running IntegrationTestIngest with a
> slowDeterministic CM, I forgot to set up SSH properly and saw the following:
> {code}
> 15/01/14 07:36:53 WARN hbase.ClusterManager: Remote command: ps aux | grep
> proc_regionserver | grep -v grep | tr -s ' ' | cut -d ' ' -f2 | xargs kill -s
> SIGKILL , hostname:node-5.internal failed at attempt 4. Retrying until
> maxAttempts: 5. Exception: stderr: Permission denied, please try again.
> Permission denied, please try again.
> Permission denied (publickey,password).
> , stdout:
> 15/01/14 07:36:53 INFO util.RetryCounter: Sleeping 16000ms before retry #4...
> 15/01/14 07:36:53 INFO zookeeper.ZooKeeper: Session: 0x14ae74d7bac006b closed
> 15/01/14 07:36:53 INFO policies.Policy: Sleeping for: 59541
> 15/01/14 07:36:53 INFO zookeeper.ClientCnxn: EventThread shut down
> Failed to write keys: 0
> Key range: [15..15]
> Batch updates: false
> Percent of keys to update: 60
> Updater threads: 10
> Ignore nonce conflicts: true
> Regions per server: 5
> 15/01/14 07:36:56 INFO util.LoadTestTool: Starting to mutate data...
> Starting to mutate data...
> 15/01/14 07:36:57 INFO policies.Policy: Sleeping for: 88816
> 15/01/14 07:37:01 INFO util.MultiThreadedAction: [U:10] Keys=471, cols=5.7 K,
> time=00:00:05 Overall: [keys/s= 94, latency=102 ms] Current: [keys/s=94,
> latency=102 ms], wroteUpTo=14
> 15/01/14 07:37:06 INFO util.MultiThreadedAction: [U:10] Keys=908, cols=11.0
> K, time=00:00:10 Overall: [keys/s= 90, latency=90 ms] Current: [keys/s=87,
> latency=77 ms], wroteUpTo=14
> 15/01/14 07:37:09 INFO hbase.ClusterManager: Executing remote command: ps aux
> | grep proc_regionserver | grep -v grep | tr -s ' ' | cut -d ' ' -f2 | xargs
> kill -s SIGKILL , hostname:node-5.internal
> 15/01/14 07:37:09 INFO util.Shell: Executing full command [/usr/bin/ssh
> node-5.internal "ps aux | grep proc_regionserver | grep -v grep | tr -s ' ' |
> cut -d ' ' -f2 | xargs kill -s SIGKILL"]
> 15/01/14 07:37:09 WARN policies.Policy: Exception occured during performing
> action: ExitCodeException exitCode=255: stderr: Permission denied, please try
> again.
> Permission denied, please try again.
> Permission denied (publickey,password).
> , stdout:
> at
> org.apache.hadoop.hbase.HBaseClusterManager.exec(HBaseClusterManager.java:208)
> at
> org.apache.hadoop.hbase.HBaseClusterManager.execWithRetries(HBaseClusterManager.java:223)
> at
> org.apache.hadoop.hbase.HBaseClusterManager.signal(HBaseClusterManager.java:268)
> at org.apache.hadoop.hbase.ClusterManager.kill(ClusterManager.java:97)
> at
> org.apache.hadoop.hbase.DistributedHBaseCluster.killRegionServer(DistributedHBaseCluster.java:110)
> at org.apache.hadoop.hbase.chaos.actions.Action.killRs(Action.java:84)
> at
> org.apache.hadoop.hbase.chaos.actions.RestartActionBaseAction.restartRs(RestartActionBaseAction.java:50)
> at
> org.apache.hadoop.hbase.chaos.actions.RestartRsHoldingMetaAction.perform(RestartRsHoldingMetaAction.java:38)
> at
> org.apache.hadoop.hbase.chaos.policies.DoActionsOncePolicy.runOneIteration(DoActionsOncePolicy.java:50)
> at
> org.apache.hadoop.hbase.chaos.policies.PeriodicPolicy.run(PeriodicPolicy.java:41)
> at
> org.apache.hadoop.hbase.chaos.policies.CompositeSequentialPolicy.run(CompositeSequentialPolicy.java:42)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Seems to me that tests should fail in these instances rather than just toss a
> warning. Was this just an oversight, [~enis] and [~ndimiduk], or is this by
> design?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
