[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210858#comment-16210858 ] Anoop Sam John commented on HBASE-15172: bq.normal HFiles, if there's special configuration for the CF in HCD, it will follow HCD. If not, it will follow the "hbase.hstore.block.storage.policy" setting in hbase-site.xml Fine. so hbase.hstore.block.storage.policy. is the one only for bulk loading. There also why it can not get the policy specific for the HCD (if any) from the corresponding Store or HCD? No way to access? My Q is is this config only for bulk loading was really needed. > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Fix For: 2.0.0 > > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210776#comment-16210776 ] Yu Li commented on HBASE-15172: --- bq. So this only for specifying policy to be used for files loaded via bulk load only? Correct. bq. For normal HFiles being flushed/compacted, the setting is at HCD? For normal HFiles, if there's special configuration for the CF in HCD, it will follow HCD. If not, it will follow the "hbase.hstore.block.storage.policy" setting in hbase-site.xml > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Fix For: 2.0.0 > > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210601#comment-16210601 ] Anoop Sam John commented on HBASE-15172: bq.we could set storage policy through "hbase.hstore.block.storage.policy" property for bulkload, or "hbase.hstore.block.storage.policy." for a specified family So this only for specifying policy to be used for files loaded via bulk load only? For normal HFiles being flushed/compacted, the setting is at HCD? > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Fix For: 2.0.0 > > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804802#comment-15804802 ] Hudson commented on HBASE-15172: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2269 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2269/]) HBASE-15172 Support setting storage policy in bulkload (liyu: rev 629b04f44f19b9589c9bcfb84da0cf5e0d4d1f18) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Fix For: 2.0.0 > > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804228#comment-15804228 ] Yu Li commented on HBASE-15172: --- Thanks for review [~tedyu] and [~ashish singhi], will commit this soon. > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801353#comment-15801353 ] Ashish Singhi commented on HBASE-15172: --- +1 > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[ https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801128#comment-15801128 ] Ted Yu commented on HBASE-15172: +1 > Support setting storage policy in bulkload > -- > > Key: HBASE-15172 > URL: https://issues.apache.org/jira/browse/HBASE-15172 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-15172.patch, HBASE-15172.v2.patch > > > When using tiered HFile storage, we should be able to generating hfile with > correct storage type during bulkload. This JIRA is targeting at making it > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[
https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800609#comment-15800609
]
Hadoop QA commented on HBASE-15172:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 30s {color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 84m 13s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 122m 48s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12845715/HBASE-15172.v2.patch |
| JIRA Issue | HBASE-15172 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux d2da9f67897d 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24
10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 20a7ae2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/5142/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/5142/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Support setting storage policy in bulkload
> --
>
> Key: HBASE-15172
> URL: https://issues.apache.org/jira/browse/HBASE-15172
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15172.patch, HBASE-15172.v2.patch
>
>
> When using tiered HFile storage, we should be able to generating hfile with
> correct storage type
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[
https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15117410#comment-15117410
]
Ted Yu commented on HBASE-15172:
{code}
357 public static void configureStoragePolicy(final Configuration conf,
final FileSystem fs,
{code}
This method can be package private
> Support setting storage policy in bulkload
> --
>
> Key: HBASE-15172
> URL: https://issues.apache.org/jira/browse/HBASE-15172
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15172.patch
>
>
> When using tiered HFile storage, we should be able to generating hfile with
> correct storage type during bulkload. This JIRA is targeting at making it
> possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15172) Support setting storage policy in bulkload
[
https://issues.apache.org/jira/browse/HBASE-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15118560#comment-15118560
]
Yu Li commented on HBASE-15172:
---
Thanks for the quick review Ted. The {{configureStoragePolicy}} method is
invoked from TestHFileOutputFormat2 in a different package, so we make it
public.
> Support setting storage policy in bulkload
> --
>
> Key: HBASE-15172
> URL: https://issues.apache.org/jira/browse/HBASE-15172
> Project: HBase
> Issue Type: Sub-task
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15172.patch
>
>
> When using tiered HFile storage, we should be able to generating hfile with
> correct storage type during bulkload. This JIRA is targeting at making it
> possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
