[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762473#comment-16762473
]
Ayush Saxena commented on HDFS-14139:
-
Yes, Correct This shall be incompatible only and we should refrain from doing
such changes. Not only scripts it can break someones use case too and we don't
have any assertions to proof that this change would be helpful in any way
.Thanks [~vinayrpet] for clarifying.
Resolving
> FsShell ls and stat command return different Modification Time on display.
> --
>
> Key: HDFS-14139
> URL: https://issues.apache.org/jira/browse/HDFS-14139
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, shell
>Reporter: Fred Peng
>Assignee: Ayush Saxena
>Priority: Major
> Labels: easyfix
> Attachments: HDFS-14139-01.patch, HDFS-14139-02.patch
>
>
> When we run "hdfs dfs -ls" or "hdfs dfs -stat" on the same file/directory,
> the time of results are different.
> Like this:
> >> $ ./hdfs dfs -stat /user/xxx/collie-pt-canary
> >> 2018-12-10 10:04:57
> >> ./hdfs dfs -ls /user/xxx/collie-pt-canary
> >> -rw-r--r-- 3 xxx supergroup 0 2018-12-10 18:04
> Strangely, we found the time is different(8 hours). The stat command uses UTC
> timezone, but the Ls command uses system local timezone.
> Why does the stat command use UTC timezone, but Ls not?
> {code:java}
> // in Stat.java
> timeFmt = new SimpleDateFormat("-MM-dd HH:mm:ss");
> timeFmt.setTimeZone(TimeZone.getTimeZone("UTC"));{code}
> By the way, in Unix/Linux the ls and stat return the same time on display.
> Should we unify the timezone?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762462#comment-16762462
]
Vinayakumar B commented on HDFS-14139:
--
Its not good idea change the output of a shell command. It might break lot of
scripts created based on Shell Command output.
Unfortunately, we dont know the reason behind the different timezones. But its
clear in the 'stat' usage that it shows UTC date/time.
> FsShell ls and stat command return different Modification Time on display.
> --
>
> Key: HDFS-14139
> URL: https://issues.apache.org/jira/browse/HDFS-14139
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, shell
>Reporter: Fred Peng
>Assignee: Ayush Saxena
>Priority: Major
> Labels: easyfix
> Attachments: HDFS-14139-01.patch, HDFS-14139-02.patch
>
>
> When we run "hdfs dfs -ls" or "hdfs dfs -stat" on the same file/directory,
> the time of results are different.
> Like this:
> >> $ ./hdfs dfs -stat /user/xxx/collie-pt-canary
> >> 2018-12-10 10:04:57
> >> ./hdfs dfs -ls /user/xxx/collie-pt-canary
> >> -rw-r--r-- 3 xxx supergroup 0 2018-12-10 18:04
> Strangely, we found the time is different(8 hours). The stat command uses UTC
> timezone, but the Ls command uses system local timezone.
> Why does the stat command use UTC timezone, but Ls not?
> {code:java}
> // in Stat.java
> timeFmt = new SimpleDateFormat("-MM-dd HH:mm:ss");
> timeFmt.setTimeZone(TimeZone.getTimeZone("UTC"));{code}
> By the way, in Unix/Linux the ls and stat return the same time on display.
> Should we unify the timezone?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722741#comment-16722741
]
Fred Peng commented on HDFS-14139:
--
Thanks [~ayushtkn] for the patch. (y) I agree its to be consistent.
I try to find the desgin reason for this issue, but still unknow. Or is it
just no reason?
What do you think?
> FsShell ls and stat command return different Modification Time on display.
> --
>
> Key: HDFS-14139
> URL: https://issues.apache.org/jira/browse/HDFS-14139
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, shell
>Reporter: Fred Peng
>Assignee: Ayush Saxena
>Priority: Major
> Labels: easyfix
> Attachments: HDFS-14139-01.patch, HDFS-14139-02.patch
>
>
> When we run "hdfs dfs -ls" or "hdfs dfs -stat" on the same file/directory,
> the time of results are different.
> Like this:
> >> $ ./hdfs dfs -stat /user/xxx/collie-pt-canary
> >> 2018-12-10 10:04:57
> >> ./hdfs dfs -ls /user/xxx/collie-pt-canary
> >> -rw-r--r-- 3 xxx supergroup 0 2018-12-10 18:04
> Strangely, we found the time is different(8 hours). The stat command uses UTC
> timezone, but the Ls command uses system local timezone.
> Why does the stat command use UTC timezone, but Ls not?
> {code:java}
> // in Stat.java
> timeFmt = new SimpleDateFormat("-MM-dd HH:mm:ss");
> timeFmt.setTimeZone(TimeZone.getTimeZone("UTC"));{code}
> By the way, in Unix/Linux the ls and stat return the same time on display.
> Should we unify the timezone?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722499#comment-16722499
]
Hadoop QA commented on HDFS-14139:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 13s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 2 new + 54 unchanged - 2 fixed = 56 total (was 56) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 57s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 6s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.util.TestDiskCheckerWithDiskIo |
| | hadoop.util.TestReadWriteDiskValidator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14139 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951939/HDFS-14139-02.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 273cef5b560f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 04c0347 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/25816/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722470#comment-16722470
]
Hadoop QA commented on HDFS-14139:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
24s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 47s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 3 new + 54 unchanged - 2 fixed = 57 total (was 56) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 13s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.ssl.TestSSLFactory |
| | hadoop.cli.TestCLI |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14139 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12951934/HDFS-14139-01.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0e2a85d9b28f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 04c0347 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/25815/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/25815/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
|
[jira] [Commented] (HDFS-14139) FsShell ls and stat command return different Modification Time on display.
[
https://issues.apache.org/jira/browse/HDFS-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722432#comment-16722432
]
Ayush Saxena commented on HDFS-14139:
-
Thanx [~pengmq1] for putting this up.verified the scenario at the linux end. It
is inline with what you mentioned.
IMO too Its better to be consistent at both places.
Have uploaded v1 with the fix.
> FsShell ls and stat command return different Modification Time on display.
> --
>
> Key: HDFS-14139
> URL: https://issues.apache.org/jira/browse/HDFS-14139
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: fs, shell
>Reporter: Fred Peng
>Assignee: Ayush Saxena
>Priority: Major
> Labels: easyfix
> Attachments: HDFS-14139-01.patch
>
>
> When we run "hdfs dfs -ls" or "hdfs dfs -stat" on the same file/directory,
> the time of results are different.
> Like this:
> >> $ ./hdfs dfs -stat /user/xxx/collie-pt-canary
> >> 2018-12-10 10:04:57
> >> ./hdfs dfs -ls /user/xxx/collie-pt-canary
> >> -rw-r--r-- 3 xxx supergroup 0 2018-12-10 18:04
> Strangely, we found the time is different(8 hours). The stat command uses UTC
> timezone, but the Ls command uses system local timezone.
> Why does the stat command use UTC timezone, but Ls not?
> {code:java}
> // in Stat.java
> timeFmt = new SimpleDateFormat("-MM-dd HH:mm:ss");
> timeFmt.setTimeZone(TimeZone.getTimeZone("UTC"));{code}
> By the way, in Unix/Linux the ls and stat return the same time on display.
> Should we unify the timezone?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
