[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520383#comment-15520383 ] Hudson commented on HBASE-16677: SUCCESS: Integrated in Jenkins build HBase-Trunk_matrix #1669 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1669/]) HBASE-16677 Add table size (total store file size) to table page (chenheng: rev f5351e2dbd29ab32dbd4044844feb6a94d9fea98) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp Revert "HBASE-16677 Add table size (total store file size) to table (chenheng: rev b14fb14886686d3135f718ff7e067230ff7d62fc) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp HBASE-16677 Add table size (total store file size) to table page (Guang (chenheng: rev f7bb6fbf21a6a86700b8411311343f0be80ebf3f) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Assignee: Guang Yang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520263#comment-15520263 ] Hudson commented on HBASE-16677: FAILURE: Integrated in Jenkins build HBase-1.4 #431 (See [https://builds.apache.org/job/HBase-1.4/431/]) HBASE-16677 Add table size (total store file size) to table page (Guang (chenheng: rev d6f9eab4be16dee0dd45ea9f1e0dd17ef8a74b7b) * (edit) hbase-server/src/main/resources/hbase-webapps/master/table.jsp > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Assignee: Guang Yang >Priority: Minor > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520071#comment-15520071 ] Guang Yang commented on HBASE-16677: Thanks [~chenheng]. > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.1.7, 1.2.4 > > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15520064#comment-15520064 ] Heng Chen commented on HBASE-16677: --- works locally. Push to master and branch-1 > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.1.7, 1.2.4 > > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15518100#comment-15518100 ] Heng Chen commented on HBASE-16677: --- LGTM. +1 Let me test it locally and commit it. Thanks [~yguang11] > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516933#comment-15516933 ] Guang Yang commented on HBASE-16677: [~chenheng], screenshot attached, thanks. > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png, table_page_v3.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[
https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515249#comment-15515249
]
Hadoop QA commented on HBASE-16677:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
14s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 30s
{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 49s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 24s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 133m 43s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient |
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures
|
| | org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache |
| |
org.apache.hadoop.hbase.replication.regionserver.TestReplicationWALReaderManager
|
| | org.apache.hadoop.hbase.master.procedure.TestRestoreSnapshotProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:7bda515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12829966/HBASE-16677_v3.patch |
| JIRA Issue | HBASE-16677 |
| Optional Tests | asflicense javac javadoc unit |
| uname | Linux 589aeb7283bf 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 07ed155 |
| Default Java | 1.8.0_101 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/3676/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-HBASE-Build/3676/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/3676/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3676/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add table size (total store file size) to table page
>
>
> Key: HBASE-16677
> URL: https://issues.apache.org/jira/browse/HBASE-16677
> Project: HBase
> Issue Type: New Feature
> Components: website
>Reporter: Guang Yang
>Priority: Minor
> Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch,
>
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[
https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515236#comment-15515236
]
Heng Chen commented on HBASE-16677:
---
+1 for patch v3, Not see regions hidden in your upload pic?
{code}
+ This table has <%= numRegions %> regions in total, in order to
improve the page load time,
+ only <%= numRegionsRendered %> regions are displayed here, click
+ here to see all regions.
+<% } %>
{code}
I mean this hit in your pic.
> Add table size (total store file size) to table page
>
>
> Key: HBASE-16677
> URL: https://issues.apache.org/jira/browse/HBASE-16677
> Project: HBase
> Issue Type: New Feature
> Components: website
>Reporter: Guang Yang
>Priority: Minor
> Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch,
> HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png,
> prod_cluster_partial.png
>
>
> Currently there is not an easy way to get the table size from the web UI,
> though we have the region size on the page, it is still convenient to have a
> table for the table size stat.
> Another pain point is that when the table grow large with tens of thousands
> of regions, it took extremely long time to load the page, however, sometimes
> we don't want to check all the regions. An optimization could be to accept a
> query parameter to specify the number of regions to render.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514989#comment-15514989 ] Guang Yang commented on HBASE-16677: [~enis], sure updated changing to use the utility. Thanks. > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, HBASE-16677_v3.patch, mini_cluster_master.png, > prod_cluster_partial.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514888#comment-15514888 ] Enis Soztutar commented on HBASE-16677: --- bq. It does not seem fit since here the store file is already MB, and in order to leverage it, we will need to convert it back to B and then call that method? Yes. It will be consistent with all human-readable sizes that we use in the UI. > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch, > HBASE-16677_v2.patch, mini_cluster_master.png, prod_cluster_partial.png > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[
https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514732#comment-15514732
]
Guang Yang commented on HBASE-16677:
Thanks [~enis] for looking at the patch.
{quote}
Do we actually stop rendering here? I did not see it. Have you tested with more
than 10K regions?
{quote}
The rendering part is in the *if* block, thus if it reaches to the threshold,
it would stop rendering more regions. I didn't test it with 10K but I tested
with query parameter like "numRegions=10" and it truncate the number of
regions to 10 in this case.
{quote}
Please use TraditionalBinaryPrefix.long2String() instead of this:
{quote}
It does not seem fit since here the store file is already MB, and in order to
leverage it, we will need to convert it back to B and then call that method?
{quote}
Can we please change the param name from regions to numRegions, and also have a
link in the warning message to the same page with the parameter set to render
all regions.
{quote}
Fixed, thanks.
> Add table size (total store file size) to table page
>
>
> Key: HBASE-16677
> URL: https://issues.apache.org/jira/browse/HBASE-16677
> Project: HBase
> Issue Type: New Feature
> Components: website
>Reporter: Guang Yang
>Priority: Minor
> Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch,
> HBASE-16677_v2.patch, mini_cluster_master.png, prod_cluster_partial.png
>
>
> Currently there is not an easy way to get the table size from the web UI,
> though we have the region size on the page, it is still convenient to have a
> table for the table size stat.
> Another pain point is that when the table grow large with tens of thousands
> of regions, it took extremely long time to load the page, however, sometimes
> we don't want to check all the regions. An optimization could be to accept a
> query parameter to specify the number of regions to render.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[
https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514062#comment-15514062
]
Enis Soztutar commented on HBASE-16677:
---
This seems useful.
Do we actually stop rendering here? I did not see it. Have you tested with more
than 10K regions?
{code}
+if (numRegionsRendered < numRegionsToRender) {
+ numRegionsRendered++;
{code}
Please use {{TraditionalBinaryPrefix.long2String()}} instead of this:
{code}
while (storeSize > 1024 && i < sizeUnits.length - 1) {
+storeSize /= 1024;
+++i;
+ }
{code}
Can we please change the param name from regions to numRegions, and also have a
link in the warning message to the same page with the parameter set to render
all regions.
> Add table size (total store file size) to table page
>
>
> Key: HBASE-16677
> URL: https://issues.apache.org/jira/browse/HBASE-16677
> Project: HBase
> Issue Type: New Feature
> Components: website
>Reporter: Guang Yang
>Priority: Minor
> Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch,
> mini_cluster_master.png, prod_cluster_partial.png
>
>
> Currently there is not an easy way to get the table size from the web UI,
> though we have the region size on the page, it is still convenient to have a
> table for the table size stat.
> Another pain point is that when the table grow large with tens of thousands
> of regions, it took extremely long time to load the page, however, sometimes
> we don't want to check all the regions. An optimization could be to accept a
> query parameter to specify the number of regions to render.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[
https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514031#comment-15514031
]
Guang Yang commented on HBASE-16677:
Thanks [~chenheng] for looking into the patch.
{quote}
Could we add one value something like "all" for regionsParam to list all
regions?
{quote}
Good point, added, thanks.
> Add table size (total store file size) to table page
>
>
> Key: HBASE-16677
> URL: https://issues.apache.org/jira/browse/HBASE-16677
> Project: HBase
> Issue Type: New Feature
> Components: website
>Reporter: Guang Yang
>Priority: Minor
> Attachments: HBASE-16677_v0.patch, HBASE-16677_v1.patch,
> mini_cluster_master.png, prod_cluster_partial.png
>
>
> Currently there is not an easy way to get the table size from the web UI,
> though we have the region size on the page, it is still convenient to have a
> table for the table size stat.
> Another pain point is that when the table grow large with tens of thousands
> of regions, it took extremely long time to load the page, however, sometimes
> we don't want to check all the regions. An optimization could be to accept a
> query parameter to specify the number of regions to render.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16677) Add table size (total store file size) to table page
[ https://issues.apache.org/jira/browse/HBASE-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512545#comment-15512545 ] Heng Chen commented on HBASE-16677: --- That is a good point. We have the same pain on production cluster (One table have 10K+ regions). Could we add one value something like "all" for regionsParam to list all regions? Upload the pic how table page looks like with the patch? > Add table size (total store file size) to table page > > > Key: HBASE-16677 > URL: https://issues.apache.org/jira/browse/HBASE-16677 > Project: HBase > Issue Type: New Feature > Components: website >Reporter: Guang Yang >Priority: Minor > Attachments: HBASE-16677_v0.patch > > > Currently there is not an easy way to get the table size from the web UI, > though we have the region size on the page, it is still convenient to have a > table for the table size stat. > Another pain point is that when the table grow large with tens of thousands > of regions, it took extremely long time to load the page, however, sometimes > we don't want to check all the regions. An optimization could be to accept a > query parameter to specify the number of regions to render. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
