[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[
https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163808#comment-17163808
]
Hemanth Boyina commented on HDFS-15473:
---
thanks for the review [~weichiu]
{quote}We don't have a counter for total number snapshots
{quote}
numSnapshots in SnapshotManager could give the total number of snapshots
{quote}Ideally, you want to use an iterator to return partial results instead
of returning all in an array
{quote}
yes , iterator will be more beneficial here , But NN dont have list for
snapshots, so adding a TreeMap in Snapshot Manager should be fine ?
> Add listSnapshots command to list all snapshots
> ---
>
> Key: HDFS-15473
> URL: https://issues.apache.org/jira/browse/HDFS-15473
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Hemanth Boyina
>Assignee: Hemanth Boyina
>Priority: Major
> Attachments: HDFS-15473.001.patch
>
>
> In a cluster where snapshots are highly used , it will benefit to have a
> command to list all the snapshots under root
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[ https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163753#comment-17163753 ] Wei-Chiu Chuang commented on HDFS-15473: bq. Also, can you add a metrics for the number of total snapshots? Hm not sure if it's a good idea since it's O(n). We don't have a counter for total number snapshots. > Add listSnapshots command to list all snapshots > --- > > Key: HDFS-15473 > URL: https://issues.apache.org/jira/browse/HDFS-15473 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Attachments: HDFS-15473.001.patch > > > In a cluster where snapshots are highly used , it will benefit to have a > command to list all the snapshots under root -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[ https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163723#comment-17163723 ] Wei-Chiu Chuang commented on HDFS-15473: Ideally, you want to use an iterator to return partial results instead of returning all in an array. The most number of snapshots in a production cluster that I have seen is ~ 10k range. Returning a few thousand snapshot entries may cause a short burst of requests in queue. Also, can you add a metrics for the number of total snapshots? > Add listSnapshots command to list all snapshots > --- > > Key: HDFS-15473 > URL: https://issues.apache.org/jira/browse/HDFS-15473 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Attachments: HDFS-15473.001.patch > > > In a cluster where snapshots are highly used , it will benefit to have a > command to list all the snapshots under root -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[
https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163089#comment-17163089
]
Hadoop QA commented on HDFS-15473:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue} 0m
0s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 1s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
18s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
35s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 4m 17s{color}
| {color:red} hadoop-hdfs-project generated 1 new + 753 unchanged - 0 fixed =
754 total (was 753) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 2s{color} | {color:orange} hadoop-hdfs-project: The patch generated 8 new +
502 unchanged - 0 fixed = 510 total (was 502) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
34s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
14s{color} | {color:green} The patch generated 0 new + 104 unchanged - 132
fixed = 104 total (was 236) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 33s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
33s{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs-client generated 3 new
+ 97 unchanged - 3 fixed = 100 total (was 100) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 7s{color}
| {color:red} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}121m 27s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
|
[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[ https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159733#comment-17159733 ] Hemanth Boyina commented on HDFS-15473: --- thanks for the comment [~weichiu] yes this command will list all snapshots s0,s1,s2,s3 , along with snapshots we list the directories in which snapshots are created in like /tmp - s0 /tmp - s1 /user - s2 /user - s3 > Add listSnapshots command to list all snapshots > --- > > Key: HDFS-15473 > URL: https://issues.apache.org/jira/browse/HDFS-15473 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > > In a cluster where snapshots are highly used , it will benefit to have a > command to list all the snapshots under root -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15473) Add listSnapshots command to list all snapshots
[ https://issues.apache.org/jira/browse/HDFS-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17159399#comment-17159399 ] Wei-Chiu Chuang commented on HDFS-15473: [~hemanthboyina] can you make an example? Say the cluster has snapshottable directories /tmp and /user, and they have snapshots s0, s1 and s2, s3 respectively. Is this command going to list all snapshots s0, s1, s2 and s3? > Add listSnapshots command to list all snapshots > --- > > Key: HDFS-15473 > URL: https://issues.apache.org/jira/browse/HDFS-15473 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > > In a cluster where snapshots are highly used , it will benefit to have a > command to list all the snapshots under root -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
