[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15119947#comment-15119947
]
Jonathan Hsieh commented on HBASE-15073:
The 1.x releases are supposed to be rock solid, and only ideally only add
features that have gone through some bake time and hardening. This concern was
brought up with a long list of other features on previous versions as well --
for example, mob, distributed log reply, snapshots, and replication.
I'm much for receptive to an argument for on by default in 2.x/master -- there
new defaults and bigger changes are to be expected.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15113379#comment-15113379
]
Ted Yu commented on HBASE-15073:
It has been uneventful 2 days where the number of regions for METRIC_RECORD
table has been stable.
Nothing interesting showed up in AMS hbase master log.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109018#comment-15109018
]
Ted Yu commented on HBASE-15073:
Here is table description where FIFO compaction and normalization are enabled:
{code}
'METRIC_RECORD', {TABLE_ATTRIBUTES => {NORMALIZATION_ENABLED => 'true',
coprocessor$1 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|805306366|', coprocessor$2
=>
'|org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver|805306366|',
coprocessor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|805306366|',
coprocessor$4 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|805306366|',
coprocessor$5 =>
'|org.apache.phoenix.hbase.index.Indexer|805306366|index.builder=org.apache.phoenix.index.PhoenixIndexBuilder,org.apache.hadoop.hbase.index.codec.class=org.apache.phoenix.index.PhoenixIndexCodec',
CONFIGURATION => {'hbase.hstore.defaultengine.compactionpolicy.class' =>
'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
'hbase.hstore.blockingStoreFiles' => '1000'}}, {NAME => '0',
DATA_BLOCK_ENCODING => 'FAST_DIFF', COMPRESSION => 'SNAPPY', TTL => '86400
SECONDS (1 DAY)'}
{code}
Will be back with some results.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15109400#comment-15109400
]
Ted Yu commented on HBASE-15073:
With DEBUG logging on, I observed interleaving region splitting / merge
operations.
e.g.
{code}
2016-01-20 13:09:48,432 INFO
[perf-a-2.c.pramod-thangali.internal,61300,1453265375538_ChoreService_1]
normalizer.SplitNormalizationPlan: Executing splitting normalization plan:
SplitNormalizationPlan{regionInfo={ENCODED => be05e5ebc6d4c10a1df4846c654bb047,
NAME =>
'METRIC_RECORD,bytes_out\x00perf-b-5.c.pramod-thangali.internal\x00\x00\x00\x01R#\xD8\x7F8HOST,1453267789291.be05e5ebc6d4c10a1df4846c654bb047.',
STARTKEY =>
'bytes_out\x00perf-b-5.c.pramod-thangali.internal\x00\x00\x00\x01R#\xD8\x7F8HOST',
ENDKEY =>
'cpu_intr\x00perf-c-386.c.pramod-thangali.internal\x00\x00\x00\x01RU\xBD;yHOST'},
splitPoint=null}
...
2016-01-20 13:19:48,437 INFO
[perf-a-2.c.pramod-thangali.internal,61300,1453265375538_ChoreService_1]
normalizer.MergeNormalizationPlan: Executing merging normalization plan:
MergeNormalizationPlan{firstRegion={ENCODED =>
0115dfae93a4974e09cea8acaa62e69b, NAME =>
'METRIC_RECORD,regionserver.Server.FlushTime_num_ops\x00perf-c-477.c.pramod-thangali.internal\x00\x00\x00\x01R\xF3dhbase,1452432082120.0115dfae93a4974e09cea8acaa62e69b.',
STARTKEY =>
'regionserver.Server.FlushTime_num_ops\x00perf-c-477.c.pramod-thangali.internal\x00\x00\x00\x01R\xF3dhbase',
ENDKEY =>
'regionserver.Server.Mutate_min\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'},
secondRegion={ENCODED => be8228be1ed0cb686bbc5d6958c5c2f1, NAME =>
'METRIC_RECORD,regionserver.Server.Mutate_min\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1452372244525.be8228be1ed0cb686bbc5d6958c5c2f1.',
STARTKEY =>
'regionserver.Server.Mutate_min\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
ENDKEY =>
'regionserver.Server.blockedRequestCount\x00perf-c-48.c.pramod-thangali.internal\x00\x00\x00\x01R!%\xA8=hbase'}}
{code}
So far the number of regions for METRIC_RECORD table is stable at 38 regions.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15107767#comment-15107767
]
Ted Yu commented on HBASE-15073:
I am doing more homework and regathering my thoughts so that I can objectively
evaluate this initiative.
This involves understanding particular use case (AMS e.g.) more deeply.
Hopefully I can come back with better answer to review comments.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15107901#comment-15107901
]
Mikhail Antonov commented on HBASE-15073:
-
I don't know how many folks are running branch-1.2 builds under any realistic
workloads, so I guess until 1.2 is out and people start to actually use it, we
are unlikely to see reports from live clusters with highlights of shortcomings
of current impl. So I think at the moment the implemented (and proposed)
heuristics are assumptions, not battle-tested. Which kind of makes me wish to
make it on by default in 1.2..
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s) (see
> https://issues.apache.org/jira/browse/HBASE-13103?focusedCommentId=14366255=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14366255),
> it would be desirable to perform only one type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105613#comment-15105613
]
Ted Yu commented on HBASE-15073:
bq. why would I want the 'choice' of turning off splits?
The following description applies to time series data with FIFO compaction
policy enforced (used by Ambari Metrics Server).
If a region continues to receive write requests, ultimately it would split -
without intervention from normalizer.
On the other hand, if the region stops receiving new write request at some
point, its size would stabilize. After TTL passes for all the HFiles in the
region, FIFO compaction would make the region empty.
So there doesn't need to be split request sent from normalizer.
In fact, the split of regions not actively receiving writes increases the work
in the future because there would be merge request(s) when such regions become
empty after compaction policy kicks in.
I talked with an Ambari developer who is involved in development of Ambari
Metrics Server and verified the above observation.
bq. where you turn normalization on and it just does the right thing
autonomously
Letting region normalizer do the work autonomously is good but we're not there
yet.
This finer grained control allows various use cases benefit from region
normalizer given the current implementation.
Again, I am open to better naming of the normalization choices.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105962#comment-15105962
]
Enis Soztutar commented on HBASE-15073:
---
Maybe we can revert back to the normalization enabled, but also include two
parameters like "hbase.normalization.split.enabled" and
"hbase.normalization.merge.enabled" as advanced options. These still can be
table level options.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106015#comment-15106015
]
stack commented on HBASE-15073:
---
-1 on this change. Revert it. It is rubbish. When a proper case is made, I'll
change my vote. I can't make sense of the above (We split because region is
big... later, because of TTL, region is empty so we merge ... but this
'increases the work'... and "I talked with an Ambari dev". and again
'various use cases' but none but the mangled one specified)
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106179#comment-15106179
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.3-IT #442 (See
[https://builds.apache.org/job/HBase-1.3-IT/442/])
HBASE-15073 Revert due to different opinion on usefulness (tedyu: rev
7a096ac6cb402f53a092362fb1aa25d2a31f38fa)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
* hbase-shell/src/main/ruby/hbase/admin.rb
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106236#comment-15106236
]
stack commented on HBASE-15073:
---
bq. ...different opinion on usefulness...
No.
The -1 is because no case was made for why we need this feature. Usually folks
show up w/ some numbers or a graph or some reasoning as to why something is
needed. In here there is none of that; there is 'certain use cases' that go
unspecified and then when pushed, some rambling about ambari metrics server
with no timelines or measure of what is being addressed or fit criteria as to
what would make ambari-case happy.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106278#comment-15106278
]
Hudson commented on HBASE-15073:
FAILURE: Integrated in HBase-1.2 #511 (See
[https://builds.apache.org/job/HBase-1.2/511/])
HBASE-15073 Revert due to different opinion on usefulness (tedyu: rev
5ff65455c60dbf56fc92630e75d93d7840dde537)
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* hbase-shell/src/main/ruby/hbase/admin.rb
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106201#comment-15106201
]
Hudson commented on HBASE-15073:
FAILURE: Integrated in HBase-1.2-IT #400 (See
[https://builds.apache.org/job/HBase-1.2-IT/400/])
HBASE-15073 Revert due to different opinion on usefulness (tedyu: rev
5ff65455c60dbf56fc92630e75d93d7840dde537)
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106296#comment-15106296
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.3 #500 (See
[https://builds.apache.org/job/HBase-1.3/500/])
HBASE-15073 Revert due to different opinion on usefulness (tedyu: rev
7a096ac6cb402f53a092362fb1aa25d2a31f38fa)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15106400#comment-15106400
]
Hudson commented on HBASE-15073:
FAILURE: Integrated in HBase-Trunk_matrix #642 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/642/])
HBASE-15073 Revert due to different opinion on usefulness (tedyu: rev
eb17f74b9e45146a21584f44e7a811fedeee138f)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103546#comment-15103546
]
Ted Yu commented on HBASE-15073:
bq. certain use case(s)
One use case is for time series data where FIFO compaction policy is enforced.
There would be regions whose data expires gradually and gets cleaned by the
compaction policy, leaving empty regions behind.
Region normalizer would issue region merge requests for the empty regions so
that the number of regions for the table is under control.
In the above scenario, region split request during normalization doesn't play
as important a role as that for region merge. Therefore user has the choice of
turning off region split (during normalization).
w.r.t. DESIRED_NORMALIZATION_TYPES, I don't see it in
hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
(master branch, branch-1 or branch-1.2)
It has been dropped since it is not referenced.
bq. Users are going to see 'MS', 'M', and 'S' and are supposed to get split,
merge, etc. from this?
Yes.
I am open to suggestion on better encoding for these two operations.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103530#comment-15103530
]
stack commented on HBASE-15073:
---
I'm trying to trace the new 'normalization' feature.
What is this about?
bq. However, for certain use case(s), it would be desirable to perform only one
type of action.
A statement about 'cases' but no cases are provided.
Why do we need finer grained control? The point is that this is an ergonomic
feature that lets the machine do it for you... if need admin intervention, then
the feature is not working right... lets fix that. This issue seems to mangle
the intent of 'normalization'.
Looking at patch I see a define: DESIRED_NORMALIZATION_TYPES What does DESIRED
mean? This is a datastore, not a beach at sunset...
Users are going to see 'MS', 'M', and 'S' and are supposed to get split, merge,
etc. from this? That's not nice on our user.
I'm -1 on this thing till its explained why. I think should revert from 1.2.
Reopening.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103608#comment-15103608
]
stack commented on HBASE-15073:
---
bq. In the above scenario, region split request during normalization doesn't
play as important a role as that for region merge.
Sure.
bq. Therefore user has the choice of turning off region split (during
normalization).
I don't follow. Would a split even be issued? The merge is keeping table
regions tidy. And then why would I want the 'choice' of turning off splits?
This is all an intellectual exercise, right? Any evidence from a running
cluster to backup need for finer grainer control?
Any other use cases other than this "If timeseries and if FIFO compaction and
if the user wants to turn off splits."?
Users don't want choice. There is too much 'choice' in hbase already. They just
want hbase to make the right choice.
bq. I am open to suggestion on better encoding for these two operations.
I suggest just purging this pursuit of 'finer grained control' over
normalization and instead try and go the other direction where you turn
normalization on and it just does the right thing autonomously... then no need
of this 'MS' or 'S' or 'M'... stuff.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086090#comment-15086090
]
Hadoop QA commented on HBASE-15073:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780773/15073-v2.txt
against master branch at commit 893a54c3a40e08b6f50d753fca01688f491893d0.
ATTACHMENT ID: 12780773
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17147//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17147//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17147//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17147//console
This message is automatically generated.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086112#comment-15086112
]
Mikhail Antonov commented on HBASE-15073:
-
I'd prefer to rename NORMALIZATION_ENABLED flag to something liek
NORMALIZATION_MODE or something similar. _ENABLED suffix strongly implies
boolean type, and using it for letter-encoded list of actions is confusing to
me. It's not yet release so we can do that; after release, it won't be until
2.0 we can change it, and likely even then there would be back pressure.
Style-wise I think it's better to return empty list rather than null from
HTD#getDesiredNormalizationTypes().
Looks good otherwise.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086130#comment-15086130
]
Ted Yu commented on HBASE-15073:
{code}
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
NonEmptyAtclauseDescriptionCheck6 7
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
IndentationCheck17 19
{code}
HBaseAdmin.java was not touched by the patch, not sure why it showed up in the
diff.
IndentationCheck warning would be covered by next patch.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085812#comment-15085812
]
Sean Busbey commented on HBASE-15073:
-
agreed that it's a good idea to get this new api straightened out before we
release it. please try to get things settled prior to HBASE-14962 being done.
as an aside, do you happen to have the jira that introduced the feature handy?
I don't recall seeing it in the list of things with release notes and want to
confirm.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15085896#comment-15085896
]
Ted Yu commented on HBASE-15073:
Linked this to HBASE-13103 where feature was first implemented
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086594#comment-15086594
]
Hadoop QA commented on HBASE-15073:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780833/15073-v5.txt
against master branch at commit a9b671b31f07ade8968b42956aa60c722032dcc8.
ATTACHMENT ID: 12780833
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17152//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17152//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17152//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17152//console
This message is automatically generated.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086486#comment-15086486
]
Hadoop QA commented on HBASE-15073:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12780826/15073-v4.txt
against master branch at commit a9b671b31f07ade8968b42956aa60c722032dcc8.
ATTACHMENT ID: 12780826
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/17150//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/17150//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/17150//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/17150//console
This message is automatically generated.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086674#comment-15086674
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.3 #483 (See
[https://builds.apache.org/job/HBase-1.3/483/])
HBASE-15073 Finer grained control over normalization actions for (tedyu: rev
53983f5b2e88812ebb5a632ee363750194ed15cb)
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086618#comment-15086618
]
Ted Yu commented on HBASE-15073:
{code}
Fetching https://builds.apache.org/job/PreCommit-HBASE-Build/17152/console
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
Printing Failing tests
Failing test : org.apache.hadoop.hbase.TestAcidGuarantees
{code}
I don't think the above test failure were caused by patch v5 which was very
close to patch v4 where unit test suite passed.
Ran the above two tests locally which also passed.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086670#comment-15086670
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.2 #492 (See
[https://builds.apache.org/job/HBase-1.2/492/])
HBASE-15073 Finer grained control over normalization actions for (tedyu: rev
8b2ea16ba1ebd4731e1512085415d29037dcd7c3)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086709#comment-15086709
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.3-IT #423 (See
[https://builds.apache.org/job/HBase-1.3-IT/423/])
HBASE-15073 Finer grained control over normalization actions for (tedyu: rev
53983f5b2e88812ebb5a632ee363750194ed15cb)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
* hbase-shell/src/main/ruby/hbase/admin.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086623#comment-15086623
]
Ted Yu commented on HBASE-15073:
{code}
./dev-support/checkstyle_report.py trunkCheckstyle.xml patchCheckstyle.xml
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
NonEmptyAtclauseDescriptionCheck6 7
{code}
Again, HBaseAdmin.java was not touched by patch v5.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086751#comment-15086751
]
Hudson commented on HBASE-15073:
SUCCESS: Integrated in HBase-1.2-IT #381 (See
[https://builds.apache.org/job/HBase-1.2-IT/381/])
HBASE-15073 Finer grained control over normalization actions for (tedyu: rev
8b2ea16ba1ebd4731e1512085415d29037dcd7c3)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizer.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/normalizer/NormalizationPlan.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/master/normalizer/TestSimpleRegionNormalizerOnCluster.java
* hbase-shell/src/main/ruby/shell/commands/normalize.rb
* hbase-client/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
* hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/MergeNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/NormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SplitNormalizationPlan.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/EmptyNormalizationPlan.java
* hbase-shell/src/main/ruby/hbase/admin.rb
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086231#comment-15086231
]
Ted Yu commented on HBASE-15073:
HTD#setNormalizationEnabled() allows user to enable normalization without
worrying about the encoding of types of action.
HTD#setNormalizationMode() is for advanced user.
What do you think ?
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086182#comment-15086182
]
Mikhail Antonov commented on HBASE-15073:
-
Method is still named setNormalizationEnabled on HTD, I'd rename it to
setNormalizationMode (and if getter is there, I'd rename it too). Also
references in admin.rb, accordingly. LGTM otherwise!
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086356#comment-15086356
]
Mikhail Antonov commented on HBASE-15073:
-
Looks good. +1 on v5.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt, 15073-v5.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15086239#comment-15086239
]
Mikhail Antonov commented on HBASE-15073:
-
I think user who may access this API is already supposed to be advanced enough
to know about split / merge normalization (provided there's javadoc for that).
I don't think we should have 2 method for essentially the same thing, one for
"less" advanced, one for "more" advanced user?
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
> Components: regionserver
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 15073-v1.txt, 15073-v2.txt, 15073-v2.txt, 15073-v3.txt,
> 15073-v4.txt
>
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15073) Finer grained control over normalization actions for RegionNormalizer
[
https://issues.apache.org/jira/browse/HBASE-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15084950#comment-15084950
]
Ted Yu commented on HBASE-15073:
w.r.t. option #1, extra work is placed on user to specify type(s) of actions.
We can interpret that the lack of this second flag means both split and merge
are allowed.
Option #2 is not flexible in that the flag doesn't give per table control.
> Finer grained control over normalization actions for RegionNormalizer
> -
>
> Key: HBASE-15073
> URL: https://issues.apache.org/jira/browse/HBASE-15073
> Project: HBase
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Ted Yu
>
> Currently both region split and merge actions are carried out during
> normalization for underlying table.
> However, for certain use case(s), it would be desirable to perform only one
> type of action.
> There is one boolean flag, keyed by NORMALIZATION_ENABLED_KEY, per table that
> enables normalization.
> To provide finer grained control, we have several options:
> 1. introduce another per table flag to indicate which type(s) of actions are
> allowed ("N" for disabled, "S" for split only, "M" for merge only and "MS"
> for both split and merge)
> 2. introduce another global flag to indicate which type(s) of actions are
> allowed
> 3. modify the meaning of existing flag keyed by NORMALIZATION_ENABLED_KEY so
> that it indicates type(s) of actions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
