[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Description: aurelius (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. was: (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > > aurelius > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Description: (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. was: (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16261) MultiHFileOutputFormat Enhancement
yi liang created HBASE-16261: Summary: MultiHFileOutputFormat Enhancement Key: HBASE-16261 URL: https://issues.apache.org/jira/browse/HBASE-16261 Project: HBase Issue Type: Sub-task Components: hbase Affects Versions: 2.0.0 Reporter: yi liang Assignee: yi liang Priority: Minor (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Description: MultiHFileOutputFormat follow HFileOutputFormat2 (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. was: aurelius (1) HFileOutputFormat2 can read one table's region split keys. and then output multiple hfiles for one family, and each hfile map to one region. We can add partitioner in MultiHFileOutputFormat to make it support this feature. (2) HFileOutputFormat2 support Customized Compression algorithm for column family and BloomFilter, also support customized DataBlockEncoding for the output hfiles. We can also make MultiHFileOutputFormat to support these features. > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386673#comment-15386673 ] yi liang commented on HBASE-16261: -- will provide patch shortly > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Attachment: HBASE-16261-V1.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Status: Patch Available (was: Open) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-16261: - Attachment: HBASE-16261-V2.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: (was: HBASE-16261-V2.patch) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBASE-16261-V2.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: (was: HBASE-16261-V2.patch) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBASE-16261-V2.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16261: - Attachment: HBASE-16261-V3.patch > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392708#comment-15392708
]
Yi Liang commented on HBASE-16234:
--
Is it fine if I pick up this jira. I have done some researches in the code, and
patch will be provided soon.
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 1.2.0
>Reporter: Harsh J
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: HBASE-16234-V1.patch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 1.2.0
>Reporter: Harsh J
> Attachments: HBASE-16234-V1.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393143#comment-15393143
]
Yi Liang commented on HBASE-16234:
--
replicaRegionsNotRecordedInMeta is used in 3 class:
1.AssignmentManager(used when master start and assign all user regions)
2.EnableTableProcedure(used when master try to enable tables)
3.EnableTableHandler(used when master try to recover tables that are not fully
moved to ENABLE state)
my idea is that if one table descriptor is null, we just skip this table and
give some warning about which table is corrupted, so that user can use hbck to
restore it, then we can enable or recover this table after restoring it.
And the patch is for master(2.0.0) branch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
> Attachments: HBASE-16234-V1.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Assignee: Yi Liang
Affects Version/s: (was: 1.2.0)
2.0.0
Status: Patch Available (was: Open)
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: HBASE-16234-V2.patch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394780#comment-15394780
]
Yi Liang commented on HBASE-16234:
--
Hi [~chenheng],
thanks for reviewing, i just read through the AssignmentManager.java file.
I found there are same problem at other places in this file, some place handle
the exception, some place not (nullpointer not belong to IOException). So in
V2, I fix all the similar problems and make all the fix consistency. Could you
take a look, thanks.
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: HBASE-16234-V3.patch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15395153#comment-15395153
]
Yi Liang commented on HBASE-16234:
--
Agreed, We'd better to use IOException. Please take a look at v3, thanks
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397145#comment-15397145
]
Yi Liang commented on HBASE-16234:
--
As the code shows, if htd is null, then the loop will continue, I just give a
warning. You mean I should stop the loop?
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397196#comment-15397196
]
Yi Liang commented on HBASE-16234:
--
public static List replicaRegionsNotRecordedInMeta(
Set regionsRecordedInMeta, MasterServices master)throws
IOException {
List regionsNotRecordedInMeta = new ArrayList();
for (HRegionInfo hri : regionsRecordedInMeta) {
TableName table = hri.getTable();
// look at the HTD for the replica count. That's the source of truth
int desiredRegionReplication = 1;
try {
HTableDescriptor htd = master.getTableDescriptors().get(table);
if (htd == null) {
LOG.warn("master can not get TableDescriptor from table '" + table);
} else {
desiredRegionReplication = htd.getRegionReplication();
}
} catch (IOException e){
LOG.warn("Couldn't get the replication attribute of the table " + table
+ " due to "
+ e.getMessage());
}
for (int i = 0; i < desiredRegionReplication; i++) {
HRegionInfo replica = RegionReplicaUtil.getRegionInfoForReplica(hri, i);
if (regionsRecordedInMeta.contains(replica)) continue;
regionsNotRecordedInMeta.add(replica);
}
}
return regionsNotRecordedInMeta;
}
ok, i got your idea, in my code, i set desiredRegionReplication==1, so if
htd==null, in the inner for() loop
the method
HRegionInfo replica = RegionReplicaUtil.getRegionInfoForReplica(hri, i);
will only get the primary replica, since every region should have primary
replica, I just want to try at least whether primary replica go to the
regionsNotRecordedInMeta list or not.
But I think you solution is also good, if the descriptor is broken, we can just
ignore this region and continue.
Which one you think is better?
and maybe we can discuss this with [~jerryhe].
Thanks :)
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398248#comment-15398248
]
Yi Liang commented on HBASE-16234:
--
Hi [~chenheng], i just check the following code in the AssignmentManager.java,
it seems that all the other place do not deal with the primary replica, all of
them skip the primary replica. So you are right, we can continue directly, I
will modify the code. Thanks for your help.
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Comment: was deleted
(was: Hi [~chenheng], i just check the following code in the
AssignmentManager.java, it seems that all the other place do not deal with the
primary replica, all of them skip the primary replica. So you are right, we can
continue directly, I will modify the code. Thanks for your help.)
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: HBASE-16234-V4.patch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch, HBASE-16234-V4.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398288#comment-15398288
]
Yi Liang commented on HBASE-16234:
--
[~chenheng], I have made changes for 4 place in the java code. i think only the
first place need to use continue, it seems that only in the first method, inner
loop for (int i = 0; i < desiredRegionReplication; i++) , start from i=0, all
the other 3 places, the inner for loop start from i=1. So I think only the
first place need to use continue, the other places only need to keep it same as
original code. What do you think?
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V1.patch,
> HBASE-16234-V2.patch, HBASE-16234-V3.patch, HBASE-16234-V4.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: (was: HBASE-16234-V1.patch)
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V2.patch,
> HBASE-16234-V3.patch, HBASE-16234-V4.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15656) Fix unused protobuf warning in Admin.proto
[
https://issues.apache.org/jira/browse/HBASE-15656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang reassigned HBASE-15656:
Assignee: Yi Liang
> Fix unused protobuf warning in Admin.proto
> --
>
> Key: HBASE-15656
> URL: https://issues.apache.org/jira/browse/HBASE-15656
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>Assignee: Yi Liang
>Priority: Minor
>
> {code}
> Warning: Unused import: "Admin.proto" imports "Client.proto" which is not
> used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15656) Fix unused protobuf warning in Admin.proto
[
https://issues.apache.org/jira/browse/HBASE-15656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399923#comment-15399923
]
Yi Liang commented on HBASE-15656:
--
{code}
yis-macbook-pro:hbase-protocol yliang$ protoc -Isrc/main/protobuf
--java_out=src/main/java src/main/protobuf/Admin.proto
[libprotobuf WARNING google/protobuf/descriptor.cc:5411] Warning: Unused
import: "Admin.proto" imports "Client.proto" which is not used.
yis-macbook-pro:hbase-protocol yliang$ protoc -Isrc/main/protobuf
--java_out=src/main/java src/main/protobuf/HBase.proto
[libprotobuf WARNING google/protobuf/descriptor.cc:5411] Warning: Unused
import: "HBase.proto" imports "Cell.proto" which is not used.
yis-macbook-pro:hbase-protocol yliang$ protoc -Isrc/main/protobuf
--java_out=src/main/java src/main/protobuf/SecureBulkLoad.proto
[libprotobuf WARNING google/protobuf/descriptor.cc:5411] Warning: Unused
import: "SecureBulkLoad.proto" imports "HBase.proto" which is not used.
yis-macbook-pro:hbase-protocol yliang$ protoc -Isrc/main/protobuf
--java_out=src/main/java src/main/protobuf/WAL.proto
[libprotobuf WARNING google/protobuf/descriptor.cc:5411] Warning: Unused
import: "WAL.proto" imports "Client.proto" which is not used.
{code}
I have made changes for this 4 proto file
> Fix unused protobuf warning in Admin.proto
> --
>
> Key: HBASE-15656
> URL: https://issues.apache.org/jira/browse/HBASE-15656
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>Priority: Minor
>
> {code}
> Warning: Unused import: "Admin.proto" imports "Client.proto" which is not
> used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15656) Fix unused protobuf warning in Admin.proto
[
https://issues.apache.org/jira/browse/HBASE-15656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-15656:
-
Attachment: HBASE-15656-V1.patch
> Fix unused protobuf warning in Admin.proto
> --
>
> Key: HBASE-15656
> URL: https://issues.apache.org/jira/browse/HBASE-15656
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>Assignee: Yi Liang
>Priority: Minor
> Attachments: HBASE-15656-V1.patch
>
>
> {code}
> Warning: Unused import: "Admin.proto" imports "Client.proto" which is not
> used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15656) Fix unused protobuf warning in Admin.proto
[
https://issues.apache.org/jira/browse/HBASE-15656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-15656:
-
Fix Version/s: 2.0.0
Status: Patch Available (was: Open)
> Fix unused protobuf warning in Admin.proto
> --
>
> Key: HBASE-15656
> URL: https://issues.apache.org/jira/browse/HBASE-15656
> Project: HBase
> Issue Type: Task
>Reporter: Elliott Clark
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15656-V1.patch
>
>
> {code}
> Warning: Unused import: "Admin.proto" imports "Client.proto" which is not
> used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15656) Fix unused protobuf warning in Admin.proto
[
https://issues.apache.org/jira/browse/HBASE-15656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-15656:
-
Affects Version/s: 2.0.0
> Fix unused protobuf warning in Admin.proto
> --
>
> Key: HBASE-15656
> URL: https://issues.apache.org/jira/browse/HBASE-15656
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Elliott Clark
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-15656-V1.patch
>
>
> {code}
> Warning: Unused import: "Admin.proto" imports "Client.proto" which is not
> used.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400724#comment-15400724
]
Yi Liang commented on HBASE-16234:
--
ok i will break out a method for those newly added code, i will name it as
getNumReplicas(TableName), which is suggested by @matteo .
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V2.patch,
> HBASE-16234-V3.patch, HBASE-16234-V4.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16234:
-
Attachment: HBASE-16234-V5.patch
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V2.patch,
> HBASE-16234-V3.patch, HBASE-16234-V4.patch, HBASE-16234-V5.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16234) Expect and handle nulls when assigning replicas
[
https://issues.apache.org/jira/browse/HBASE-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400935#comment-15400935
]
Yi Liang commented on HBASE-16234:
--
patch V5 is attached, please take a look. Thanks :)
> Expect and handle nulls when assigning replicas
> ---
>
> Key: HBASE-16234
> URL: https://issues.apache.org/jira/browse/HBASE-16234
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 2.0.0
>Reporter: Harsh J
>Assignee: Yi Liang
> Attachments: HBASE-16234-V1.patch, HBASE-16234-V2.patch,
> HBASE-16234-V3.patch, HBASE-16234-V4.patch, HBASE-16234-V5.patch
>
>
> Observed this on a cluster:
> {code}
> FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting
> shutdown.
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.AssignmentManager.replicaRegionsNotRecordedInMeta(AssignmentManager.java:2799)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assignAllUserRegions(AssignmentManager.java:2778)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.processDeadServersAndRegionsInTransition(AssignmentManager.java:638)
>
> at
> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:485)
>
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:723)
>
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1481)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> It looks like {{FSTableDescriptors#get(…)}} can be expected to return null in
> some cases, but {{AssignmentManager.replicaRegionsNotRecordedInMeta(…)}} does
> not currently have any handling for such a possibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408131#comment-15408131
]
Yi Liang commented on HBASE-16353:
--
protobuf
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408203#comment-15408203
]
Yi Liang commented on HBASE-16353:
--
I have done some research in the code, found that
1. in 2.0 branch
(1) All the getClosestRow related methods have been removed, except the one in
HBaseTestingUtility. And that one in HBaseTestUtility is implemented using
reversed scanner, so no need to remove this one.
(2) I did not found any related filed in PB, it seems the below filed has been
removed in 2.0
optional bool closest_row_before = 11 [default = false]; (which found in 1.4
client.proto)
So, for 2.0 we just need to remove Get.setClosestRowBefore() /
isClosestRowBefore, and in 1.4 we need more modify.
I will first attach the patch for 2.0 now, and if patch for 1.4 is needed, I
will attach it later.
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16353:
-
Attachment: HBASE-16353-V1.patch
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16353:
-
Assignee: Yi Liang
Fix Version/s: (was: 1.4.0)
Status: Patch Available (was: Open)
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16353:
-
Fix Version/s: 1.4.0
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16261) MultiHFileOutputFormat Enhancement
[ https://issues.apache.org/jira/browse/HBASE-16261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412552#comment-15412552 ] Yi Liang commented on HBASE-16261: -- could any one help me to review the code, Thanks. :) > MultiHFileOutputFormat Enhancement > > > Key: HBASE-16261 > URL: https://issues.apache.org/jira/browse/HBASE-16261 > Project: HBase > Issue Type: Sub-task > Components: hbase >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16261-V1.patch, HBASE-16261-V2.patch, > HBASE-16261-V3.patch > > > MultiHFileOutputFormat follow HFileOutputFormat2 > (1) HFileOutputFormat2 can read one table's region split keys. and then > output multiple hfiles for one family, and each hfile map to one region. We > can add partitioner in MultiHFileOutputFormat to make it support this feature. > (2) HFileOutputFormat2 support Customized Compression algorithm for column > family and BloomFilter, also support customized DataBlockEncoding for the > output hfiles. We can also make MultiHFileOutputFormat to support these > features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415636#comment-15415636
]
Yi Liang commented on HBASE-16385:
--
Hi [~dimaspivak],
From [HBASE-3272|https://issues.apache.org/jira/browse/HBASE-3272], you can
take a look at the patch, the code used to get {{hbase.rest.port}} from
Constant.java, but it have changed to hardcoded, do you think we need to
remove the {{DEFAULT_LISTEN_PORT}} in constant.java which no longer use, or
change it back ?
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Priority: Minor
> Labels: beginner
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang reassigned HBASE-16385:
Assignee: Yi Liang
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Assignee: Yi Liang
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-16385-V1.patch
>
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16385:
-
Attachment: HBASE-16385-V1.patch
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-16385-V1.patch
>
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415699#comment-15415699
]
Yi Liang commented on HBASE-16385:
--
I provided a patch, please take a look, thanks :)
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-16385-V1.patch
>
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16385:
-
Status: Patch Available (was: Open)
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Assignee: Yi Liang
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-16385-V1.patch
>
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16385) Have hbase-rest pull hbase.rest.port from Constants.java
[
https://issues.apache.org/jira/browse/HBASE-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16385:
-
Fix Version/s: 2.0.0
> Have hbase-rest pull hbase.rest.port from Constants.java
>
>
> Key: HBASE-16385
> URL: https://issues.apache.org/jira/browse/HBASE-16385
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Reporter: Dima Spivak
>Assignee: Yi Liang
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-16385-V1.patch
>
>
> [A while back|https://issues.apache.org/jira/browse/HBASE-3272], we
> [hardcoded|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java#L266]
> the {{hbase.rest.port}} instead of getting it from
> [Constants.java|https://github.com/apache/hbase/blob/master/hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/Constants.java#L36].
> We should fix that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424978#comment-15424978 ] Yi Liang commented on HBASE-16381: -- Hi [~apurtell], I am now working on this jira, I think we had better add a new command like 'deleterows' for this jira. Since deleteall is used for deleting all cells in a given row. It is only used for only one row, but if expend this command to delete multiple rows will make deleteall command not so consistency(one command has different functionalities), and the usage of this command will become a little complicated. so I suggest adding a new command. > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Priority: Minor > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang reassigned HBASE-16381: Assignee: Yi Liang > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427146#comment-15427146
]
Yi Liang commented on HBASE-16353:
--
@[~enis], are you ok with this patch?
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Attachment: HBASE-16381-V1.patch > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Attachments: HBASE-16381-V1.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Fix Version/s: 2.0.0 Status: Patch Available (was: Open) > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433365#comment-15433365
]
Yi Liang commented on HBASE-16381:
--
Hi [~apurtell],
I have attached a patch, which I add a new command called 'deleterows', the
usage is as below:
hbase> deleterows 'ns1:t1', 'prefix'
hbase> deleterows 't1', 'prefix' // delete all cells
for specific rows
hbase> deleterows 't1', 'prefix', 'c1' // delete specific
columns for specific rows
hbase> deleterows 't1', 'prefix', 'c1', ts1 // delete specific
columns for specific rows before ts1
hbase> deleterows 't1', 'prefix', 'c1', {VISIBILITY=>'PRIVATE|SECRET'}
hbase> deleterows 't1', 'prefix', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'}
//above two considering cell visibility
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433365#comment-15433365
]
Yi Liang edited comment on HBASE-16381 at 8/23/16 6:41 PM:
---
Hi [~apurtell],
I have attached a patch, which I add a new command called 'deleterows', the
usage is as below:
{code}
hbase> deleterows 'ns1:t1', 'prefix'
hbase> deleterows 't1', 'prefix' // delete all cells
for specific rows
hbase> deleterows 't1', 'prefix', 'c1' // delete specific
columns for specific rows
hbase> deleterows 't1', 'prefix', 'c1', ts1 // delete specific
columns for specific rows before ts1
hbase> deleterows 't1', 'prefix', 'c1', {VISIBILITY=>'PRIVATE|SECRET'}
hbase> deleterows 't1', 'prefix', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'}
//above two considering cell visibility
{code}
was (Author: easyliangjob):
Hi [~apurtell],
I have attached a patch, which I add a new command called 'deleterows', the
usage is as below:
hbase> deleterows 'ns1:t1', 'prefix'
hbase> deleterows 't1', 'prefix' // delete all cells
for specific rows
hbase> deleterows 't1', 'prefix', 'c1' // delete specific
columns for specific rows
hbase> deleterows 't1', 'prefix', 'c1', ts1 // delete specific
columns for specific rows before ts1
hbase> deleterows 't1', 'prefix', 'c1', {VISIBILITY=>'PRIVATE|SECRET'}
hbase> deleterows 't1', 'prefix', 'c1', ts1, {VISIBILITY=>'PRIVATE|SECRET'}
//above two considering cell visibility
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16311:
-
Attachment: HBASE-16311-V1.patch
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Abhishek Kumar
>Priority: Minor
> Attachments: HBASE-16311-V1.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang reassigned HBASE-16311:
Assignee: Yi Liang
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Attachments: HBASE-16311-V1.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433870#comment-15433870
]
Yi Liang commented on HBASE-16311:
--
I found that no activity for this jira for 3 weeks, so I take this jira and
provide a patch.
in my patch, i finished two parts:
(1)TODO part:
log the delete_snapshot and list_snapshot info into audti log
(2) log snapshot name into audit logs,
Since 'AuthResult'(which is used to store audit log related information)
class does not have field/variable for snapshot, so I just put the snapshot
name into the 'request' filed in AuthResult. The output is showed below:
{code}
2016-08-23 16:29:47,423 TRACE
SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: Access
allowed for user hbase; reason: Global check allowed; remote address:
/172.16.156.175; request: deleteSnapshot my-snapshot; context: (user=hbase,
scope=GLOBAL, action=ADMIN)
{code}
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Attachments: HBASE-16311-V1.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16311:
-
Fix Version/s: 2.0.0
Affects Version/s: 2.0.0
Status: Patch Available (was: Open)
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Attachment: HBASE-16381-V2.patch > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16311:
-
Attachment: HBASE-16311-V2.patch
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435269#comment-15435269
]
Yi Liang commented on HBASE-16311:
--
patch V2 carry Matteo's comments
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435274#comment-15435274 ] Yi Liang commented on HBASE-16381: -- V2 fix some warnings and format issues. > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436012#comment-15436012
]
Yi Liang commented on HBASE-16381:
--
ping [~tedyu] [~chenheng] [~apurtell] [~stack], could you guys take a look at
this jira :), I think this is a useful feature. Right now, I have some concerns
(1) do we need to add new command, usage seen last comment,
(2) or just as Andrew suggest that we can expend this deleteall command
if expand this command I think we can replace the 'row' argument in deleteall
command, shows as below.
{code}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1'
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1,
{VISIBILITY=>'PRIVATE|SECRET'}
{code}
Or you guys have any other good solutions.
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436206#comment-15436206
]
Yi Liang commented on HBASE-16381:
--
Thanks for your feedback, the current deleteall is used to delete all cells in
one row. Just like what you said, to delete rows having 'prefix' in one cell,
we can
(1)deleterows 't1', 'prefix', 'c1' > add new command
(2)deleteall 't1', {ROWPREFIXFILTER=>'prefix'}, c1 ==> extend old one
if we want to delete all rows in a cell, we may use '' empty string
(1) deleterows 't1', '', 'c1'
(2) deleteall 't1',{ROWPREFIXFILTER=>''}, 'c1'
But I think using new command is better, since deleteall is only for deleting
one row, extend it to delete multiple rows can make this command not so
consistent.
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436206#comment-15436206
]
Yi Liang edited comment on HBASE-16381 at 8/25/16 3:27 AM:
---
Thanks for your feedback, the current deleteall is used to delete all cells in
one row. Just like what you said, to delete rows having 'prefix' in one cell,
we can either
(1)deleterows 't1', 'prefix', 'c1'
(2)deleteall 't1', {ROWPREFIXFILTER=>'prefix'}, c1
if we want to delete all rows in a cell, we may use '' empty string
(1) deleterows 't1', '', 'c1'
(2) deleteall 't1',{ROWPREFIXFILTER=>''}, 'c1'
But I think using new command is better, since deleteall is only for deleting
one row, extend it to delete multiple rows can make this command not so
consistent.
was (Author: easyliangjob):
Thanks for your feedback, the current deleteall is used to delete all cells in
one row. Just like what you said, to delete rows having 'prefix' in one cell,
we can
(1)deleterows 't1', 'prefix', 'c1' > add new command
(2)deleteall 't1', {ROWPREFIXFILTER=>'prefix'}, c1 ==> extend old one
if we want to delete all rows in a cell, we may use '' empty string
(1) deleterows 't1', '', 'c1'
(2) deleteall 't1',{ROWPREFIXFILTER=>''}, 'c1'
But I think using new command is better, since deleteall is only for deleting
one row, extend it to delete multiple rows can make this command not so
consistent.
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15436206#comment-15436206
]
Yi Liang edited comment on HBASE-16381 at 8/25/16 3:27 AM:
---
Thanks for your feedback, the current deleteall is used to delete all cells in
one row. Just like what you said, to delete rows having 'prefix' in one cell,
we can either
(1)deleterows 't1', 'prefix', 'c1'
(2)deleteall 't1', {ROWPREFIXFILTER=>'prefix'}, 'c1'
if we want to delete all rows in a cell, we may use '' empty string
(1) deleterows 't1', '', 'c1'
(2) deleteall 't1',{ROWPREFIXFILTER=>''}, 'c1'
But I think using new command is better, since deleteall is only for deleting
one row, extend it to delete multiple rows can make this command not so
consistent.
was (Author: easyliangjob):
Thanks for your feedback, the current deleteall is used to delete all cells in
one row. Just like what you said, to delete rows having 'prefix' in one cell,
we can either
(1)deleterows 't1', 'prefix', 'c1'
(2)deleteall 't1', {ROWPREFIXFILTER=>'prefix'}, c1
if we want to delete all rows in a cell, we may use '' empty string
(1) deleterows 't1', '', 'c1'
(2) deleteall 't1',{ROWPREFIXFILTER=>''}, 'c1'
But I think using new command is better, since deleteall is only for deleting
one row, extend it to delete multiple rows can make this command not so
consistent.
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437171#comment-15437171
]
Yi Liang commented on HBASE-16353:
--
Just review the related code, from the comments found that they will be removed
in 3.0.0, so we can mark it as resolved later.
{code}
/**
* This will always return the default value which is false as client cannot
set the value to this
* property any more.
* @deprecated since 2.0.0 and will be removed in 3.0.0
*/
@Deprecated
public boolean isClosestRowBefore() {
return closestRowBefore;
}
/**
* This is not used any more and does nothing. Use reverse scan instead.
* @deprecated since 2.0.0 and will be removed in 3.0.0
*/
@Deprecated
public Get setClosestRowBefore(boolean closestRowBefore) {
// do Nothing
return this;
}
{code}
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16353) Deprecate / Remove Get.isClosestRowBefore()
[
https://issues.apache.org/jira/browse/HBASE-16353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16353:
-
Resolution: Later
Status: Resolved (was: Patch Available)
> Deprecate / Remove Get.isClosestRowBefore()
>
>
> Key: HBASE-16353
> URL: https://issues.apache.org/jira/browse/HBASE-16353
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Yi Liang
> Labels: beginner
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16353-V1.patch
>
>
> We no longer have "get closest row" functionality in 1.0+. We forgot to
> remove Get.setClosestRowBefore() / isClosestRowBefore and deprecate the
> field in PB {{closest_row_before}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437305#comment-15437305
]
Yi Liang commented on HBASE-16311:
--
[~mbertozzi], have a question about errors in the
hadoop.hbase.security.access.TestAccessController
in the test code, the 'snapshot' parameter in clone action has been set as
'NULL' on purpose, that's why not pass test, but all other action such as
listsnapshot, and deletesnapshot, their snapshot parameter are not set null.
However, in its original code,all the Accesscontroller#preXXXSnapshot method
have not deal with situation when 'snapshot' parameter is 'NULL'
code below is in the hadoop.hbase.security.access.TestAccessController, where
the programmer set snapshot name as null in cloneAction on purpose.
{code}
2052 AccessTestAction cloneAction = new AccessTestAction() {
2053 @Override
2054 public Object run() throws Exception {
2055
ACCESS_CONTROLLER.preCloneSnapshot(ObserverContext.createAndPrepare(CP_ENV,
null),
2056 null, null);
2057return null;
2058 }
2059 };
{code}
code below is in Accesscontroller#preXXXSnapshot,where the programmer did not
deal with snapshot parameter as null, and all other preXXX are not deal with
null snapshot as well
{code}
1335 @Override
1336 public void preCloneSnapshot(final
ObserverContext ctx,
1337 final SnapshotDescription snapshot, final HTableDescriptor
hTableDescriptor)
1338 throws IOException {
1339requirePermission(getActiveUser(ctx), "clone " + snapshot.getName(),
Action.ADMIN);
1340 }
{code}
do you think I need to change the code to deal with null snapshot parameter,
which has lot of works; or I just easily change the cloneAction parameter to a
non-null snapshot in test code.
Or do you know anybody who is familiar with this part of code, we can ask his
advice. Thanks
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-15523: - Resolution: Invalid Status: Resolved (was: Patch Available) > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang resolved HBASE-15523. -- Resolution: Fixed > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: yi liang >Assignee: yi liang >Priority: Minor > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
yi liang created HBASE-15523: Summary: enhance hbase-daemon.sh to enable autorestart. Key: HBASE-15523 URL: https://issues.apache.org/jira/browse/HBASE-15523 Project: HBase Issue Type: Improvement Reporter: yi liang Assignee: yi liang Priority: Minor enhance hbase-daemon.sh to enable autorestart. component(like master, region server) will auto-start when terminated/killed abnormally if (a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. export HBASE_AUTORESTART=true (b) Then add the following 3 simple lines(59-61) to /bin/hbase-daemon.sh 51 # get arguments 52 startStop=$1 53 shift 54 55 command=$1 56 shift 57 58 #make sure the auto-restart are default settings 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; then 60 startStop="autorestart" 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15523 started by yi liang. > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: yi liang >Assignee: yi liang >Priority: Minor > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-15523: - Status: Patch Available (was: In Progress) > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-15523: - Attachment: HBASE-15523.patch > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211019#comment-15211019 ] yi liang commented on HBASE-15523: -- hi Dima Spivak, This issue come from one of my customers, they use management console like Ambari-UI to manage the hbase, and start function of Hbase in Ambari-UI is hard-coded. So every time, when hbase component like region server terminated abnormally, they have to manually restart the HBase Component. they can not tolerate with it, so they want the flexibitliy to use an env variable they can input in the console to control the behavior. > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: yi liang >Assignee: yi liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16311:
-
Attachment: HBASE-16311-V3.patch
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch,
> HBASE-16311-V3.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
Yi Liang created HBASE-16544: Summary: Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide Key: HBASE-16544 URL: https://issues.apache.org/jira/browse/HBASE-16544 Project: HBase Issue Type: Bug Components: documentation, snapshots Affects Versions: 2.0.0 Reporter: Yi Liang reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I think this title means that we can use S3 storage to replace HDFS, I really tried this :(, always give me errors and hbase even can not start, see error mentioned in jira HBASE-11045. (2) And the details in this section are more about deploying HBase on Amazon EC2 cluster, which has nothing to do 'using Amazon S3 storage' (3) In all, I think we need to remove this section, or at least clarify this section if someone fully test HBase on S3. see HBASE-15646 for more details about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456492#comment-15456492
]
Yi Liang commented on HBASE-16311:
--
thank for reviewing [~jerryhe], yes, the snapshot owner may not have global
Action.ADMIN permission, so we can just set it as null
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch,
> HBASE-16311-V3.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16311) Audit log for delete snapshot operation is missing in case of snapshot owner deleting the same
[
https://issues.apache.org/jira/browse/HBASE-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16311:
-
Attachment: HBASE-16311-V4.patch
> Audit log for delete snapshot operation is missing in case of snapshot owner
> deleting the same
> --
>
> Key: HBASE-16311
> URL: https://issues.apache.org/jira/browse/HBASE-16311
> Project: HBase
> Issue Type: Sub-task
> Components: snapshots
>Affects Versions: 2.0.0
>Reporter: Abhishek Kumar
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16311-V1.patch, HBASE-16311-V2.patch,
> HBASE-16311-V3.patch, HBASE-16311-V4.patch
>
>
> 1. Audit log seems to be left as a TODO task in AccessController.java:
> {code}
> @Override
> public void preDeleteSnapshot(final
> ObserverContext ctx,
> final SnapshotDescription snapshot) throws IOException {
> if (SnapshotDescriptionUtils.isSnapshotOwner(snapshot, getActiveUser())) {
> // Snapshot owner is allowed to delete the snapshot
> // TODO: We are not logging this for audit
> } else {
> requirePermission("deleteSnapshot", Action.ADMIN);
> }
> }
> {code}
> 2. Also, snapshot name is not getting logged in the audit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang reassigned HBASE-16544: Assignee: Yi Liang > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, snapshots >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Component/s: (was: snapshots) Filesystem Integration > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Attachment: HBASE-16544-V1.patch > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Attachments: HBASE-16544-V1.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Fix Version/s: 2.0.0 Status: Patch Available (was: Open) > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-16544-V1.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459702#comment-15459702 ] Yi Liang commented on HBASE-16544: -- Provide a patch to remove the 'Using Amazon S3 Storage', because, this title is confusing and hbase does not fully support Amazon S3, see HBASE-11045. And also 'Storing Snapshots in an Amazon S3' has been covered in section 138.9. no need to have 'using amazon s3 storage' this section in ref guide. [~misty], are you ok with this patch? > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-16544-V1.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Status: Open (was: Patch Available) > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-16544-V1.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Status: Patch Available (was: Open) > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-16544-V1.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Attachment: HBASE-16381-V3.patch > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch, > HBASE-16381-V3.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477676#comment-15477676
]
Yi Liang commented on HBASE-16381:
--
yes, agree with Jerry. You can see from below code, the delete also use
deleteall's implementation, there maybe some reason why two similar delete
commands could exist.
{code}
# Delete a cell
def _delete_internal(row, column,
timestamp =
org.apache.hadoop.hbase.HConstants::LATEST_TIMESTAMP, args = {})
_deleteall_internal(row, column, timestamp, args)
end
{code}
I provide a patch that extend deleteall to support row prefix.
the usage is also below,
{code}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1'//delete
certain cf in the row ranges
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1,
{VISIBILITY=>'PRIVATE|SECRET'}
{code}
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch,
> HBASE-16381-V3.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15477676#comment-15477676
]
Yi Liang edited comment on HBASE-16381 at 9/9/16 5:35 PM:
--
yes, agree with Jerry. You can see from below code, the delete also use
deleteall's implementation, there maybe some reason why two similar delete
commands could co-exist.
{code}
# Delete a cell
def _delete_internal(row, column,
timestamp =
org.apache.hadoop.hbase.HConstants::LATEST_TIMESTAMP, args = {})
_deleteall_internal(row, column, timestamp, args)
end
{code}
I provide a patch that extend deleteall to support row prefix.
the usage is also below,
{code}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1'//delete
certain cf in the row ranges
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1,
{VISIBILITY=>'PRIVATE|SECRET'}
{code}
was (Author: easyliangjob):
yes, agree with Jerry. You can see from below code, the delete also use
deleteall's implementation, there maybe some reason why two similar delete
commands could exist.
{code}
# Delete a cell
def _delete_internal(row, column,
timestamp =
org.apache.hadoop.hbase.HConstants::LATEST_TIMESTAMP, args = {})
_deleteall_internal(row, column, timestamp, args)
end
{code}
I provide a patch that extend deleteall to support row prefix.
the usage is also below,
{code}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1'//delete
certain cf in the row ranges
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix'}, 'c1', ts1,
{VISIBILITY=>'PRIVATE|SECRET'}
{code}
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch,
> HBASE-16381-V3.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15479270#comment-15479270
]
Yi Liang commented on HBASE-16381:
--
yes, when we want to delete multiple rows, we just need to replace 'row'
parameter with {ROWPREFIXFILTER => 'prefix'}, this is the usage.
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch,
> HBASE-16381-V3.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Attachment: HBASE-16381-V4.patch Add test in hbase shell tests > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch, > HBASE-16381-V3.patch, HBASE-16381-V4.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488278#comment-15488278 ] Yi Liang commented on HBASE-16381: -- yes, i will change it to reuse this list > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch, > HBASE-16381-V3.patch, HBASE-16381-V4.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16381: - Attachment: HBASE-16381-V5.patch > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch, > HBASE-16381-V3.patch, HBASE-16381-V4.patch, HBASE-16381-V5.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16381) Shell deleteall command should support row key prefixes
[
https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liang updated HBASE-16381:
-
Attachment: HBASE-16381-V6.patch
Carry Chen Heng's comments
{code}
hbase> deleteall 't1', {ROWPREFIXFILTER => 'prefix', CACHE => 100}
{code}
CACHE is optional, and is used to specify how many deletes batched to be sent
to server at one time, default is 100
> Shell deleteall command should support row key prefixes
> ---
>
> Key: HBASE-16381
> URL: https://issues.apache.org/jira/browse/HBASE-16381
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Andrew Purtell
>Assignee: Yi Liang
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch,
> HBASE-16381-V3.patch, HBASE-16381-V4.patch, HBASE-16381-V5.patch,
> HBASE-16381-V6.patch
>
>
> The shell's deleteall command should support deleting a row range using a row
> key prefix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16381) Shell deleteall command should support row key prefixes
[ https://issues.apache.org/jira/browse/HBASE-16381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493999#comment-15493999 ] Yi Liang commented on HBASE-16381: -- Thanks for committing [~chenheng] and thanks for reviewing [~jerryhe] > Shell deleteall command should support row key prefixes > --- > > Key: HBASE-16381 > URL: https://issues.apache.org/jira/browse/HBASE-16381 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Andrew Purtell >Assignee: Yi Liang >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16381-V1.patch, HBASE-16381-V2.patch, > HBASE-16381-V3.patch, HBASE-16381-V4.patch, HBASE-16381-V5.patch, > HBASE-16381-V6.patch > > > The shell's deleteall command should support deleting a row range using a row > key prefix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang reopened HBASE-15523: -- > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15523) enhance hbase-daemon.sh to enable autorestart.
[ https://issues.apache.org/jira/browse/HBASE-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496861#comment-15496861 ] Yi Liang commented on HBASE-15523: -- I closed this one on Aug by accident, I reopen it to see if anyone have any other suggestion or idea of this proposal. From my point of view, the autostart command is also a kind of extension for start command, we need somehow extend start command to support this autostart, not have a new command for autostart. > enhance hbase-daemon.sh to enable autorestart. > -- > > Key: HBASE-15523 > URL: https://issues.apache.org/jira/browse/HBASE-15523 > Project: HBase > Issue Type: Improvement >Reporter: Yi Liang >Assignee: Yi Liang >Priority: Minor > Attachments: HBASE-15523.patch > > > enhance hbase-daemon.sh to enable autorestart. > component(like master, region server) will auto-start when terminated/killed > abnormally if >(a) Add a new env variable $HBASE_AUTORESTART to hbase-env.sh i.e. > export HBASE_AUTORESTART=true > (b) Then add the following 3 simple lines(59-61) to > /bin/hbase-daemon.sh > > 51 # get arguments > 52 startStop=$1 > 53 shift > 54 > 55 command=$1 > 56 shift > 57 > 58 #make sure the auto-restart are default settings > 59 if [ "$HBASE_AUTORESTART" == "true" ] && [ "$startStop" == "start" ]; > then > 60 startStop="autorestart" > 61 fi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16544) Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide
[ https://issues.apache.org/jira/browse/HBASE-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-16544: - Attachment: HBASE-16544-addendum.patch forget to remove links to the removed section 'using amazon s3'. Provide addendum to remove it. > Remove or Clarify 'Using Amazon S3 Storage' section in the reference guide > --- > > Key: HBASE-16544 > URL: https://issues.apache.org/jira/browse/HBASE-16544 > Project: HBase > Issue Type: Bug > Components: documentation, Filesystem Integration >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-16544-V1.patch, HBASE-16544-addendum.patch > > > reference guide at https://hbase.apache.org/book.html#amazon_s3_configuration > (1) the title 'Using Amazon S3 Storage' is confusing.From my point of view, I > think this title means that we can use S3 storage to replace HDFS, I really > tried this :(, always give me errors and hbase even can not start, see > error mentioned in jira HBASE-11045. > (2) And the details in this section are more about deploying HBase on Amazon > EC2 cluster, which has nothing to do 'using Amazon S3 storage' > (3) In all, I think we need to remove this section, or at least clarify this > section if someone fully test HBase on S3. see HBASE-15646 for more details > about this doc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
