[jira] [Updated] (HADOOP-17305) ITestCustomSigner fails with gcs s3 compatible endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-17305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17305: Labels: pull-request-available (was: ) > ITestCustomSigner fails with gcs s3 compatible endpoint. > - > > Key: HADOOP-17305 > URL: https://issues.apache.org/jira/browse/HADOOP-17305 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > CC [~sseth] [~ste...@apache.org] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17305) ITestCustomSigner fails with gcs s3 compatible endpoint.
[ https://issues.apache.org/jira/browse/HADOOP-17305?focusedWorklogId=502012=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502012 ] ASF GitHub Bot logged work on HADOOP-17305: --- Author: ASF GitHub Bot Created on: 19/Oct/20 05:54 Start Date: 19/Oct/20 05:54 Worklog Time Spent: 10m Work Description: mukund-thakur opened a new pull request #2395: URL: https://github.com/apache/hadoop/pull/2395 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 502012) Remaining Estimate: 0h Time Spent: 10m > ITestCustomSigner fails with gcs s3 compatible endpoint. > - > > Key: HADOOP-17305 > URL: https://issues.apache.org/jira/browse/HADOOP-17305 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > CC [~sseth] [~ste...@apache.org] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur opened a new pull request #2395: HADOOP-17305 Fix ITestCustomSigner to work with s3 compatible endpoints
mukund-thakur opened a new pull request #2395: URL: https://github.com/apache/hadoop/pull/2395 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17306) RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
[
https://issues.apache.org/jira/browse/HADOOP-17306?focusedWorklogId=502006=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502006
]
ASF GitHub Bot logged work on HADOOP-17306:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 05:30
Start Date: 19/Oct/20 05:30
Worklog Time Spent: 10m
Work Description: vinayakumarb commented on a change in pull request
#2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507475468
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
Sure. Thanks for details @aajisaka
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502006)
Time Spent: 1h 10m (was: 1h)
> RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
>
>
> Key: HADOOP-17306
> URL: https://issues.apache.org/jira/browse/HADOOP-17306
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> RawLocalFileSystem's FileStatus uses {{File.lastModified()}} api from JDK.
> This api looses milliseconds due to JDK bug.
> [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8177809]
> This bug fixed in JDK 10 b09 onwards and still exists in JDK 8 which is still
> being used in many productions.
> Apparently, {{Files.getLastModifiedTime()}} from java's nio package returns
> correct time.
> Use {{Files.getLastModifiedTime()}} instead of {{File.lastModified}} as
> workaround.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17306) RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
[
https://issues.apache.org/jira/browse/HADOOP-17306?focusedWorklogId=502005=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502005
]
ASF GitHub Bot logged work on HADOOP-17306:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 05:29
Start Date: 19/Oct/20 05:29
Worklog Time Spent: 10m
Work Description: aajisaka commented on a change in pull request #2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507473450
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
HFS+ does not support milliseconds for mtime, so I agreed to skip this
in MacOS. In contrast, APFS supports nanoseconds and JDK-8177809 is fixed if
the filesystem is APFS.
```
➜ hadoop git:(trunk) uname -a
Darwin 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT
2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64
➜ hadoop git:(trunk) jshell
| Welcome to JShell -- Version 11.0.8
| For an introduction type: /help intro
jshell> Files.getLastModifiedTime(Paths.get("./pom.xml"))
$1 ==> 2020-10-19T04:51:58.69531Z
jshell> new File("./pom.xml").lastModified()
$1 ==> 1603083118695
```
Would you update the comment like "Skip MacOS because HFS+ does not support
milliseconds for mtime." ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502005)
Time Spent: 1h (was: 50m)
> RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
>
>
> Key: HADOOP-17306
> URL: https://issues.apache.org/jira/browse/HADOOP-17306
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> RawLocalFileSystem's FileStatus uses {{File.lastModified()}} api from JDK.
> This api looses milliseconds due to JDK bug.
> [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8177809]
> This bug fixed in JDK 10 b09 onwards and still exists in JDK 8 which is still
> being used in many productions.
> Apparently, {{Files.getLastModifiedTime()}} from java's nio package returns
> correct time.
> Use {{Files.getLastModifiedTime()}} instead of {{File.lastModified}} as
> workaround.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2387: HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
vinayakumarb commented on a change in pull request #2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507475468
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
Sure. Thanks for details @aajisaka
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2387: HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
aajisaka commented on a change in pull request #2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507473450
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
HFS+ does not support milliseconds for mtime, so I agreed to skip this
in MacOS. In contrast, APFS supports nanoseconds and JDK-8177809 is fixed if
the filesystem is APFS.
```
➜ hadoop git:(trunk) uname -a
Darwin 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT
2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64
➜ hadoop git:(trunk) jshell
| Welcome to JShell -- Version 11.0.8
| For an introduction type: /help intro
jshell> Files.getLastModifiedTime(Paths.get("./pom.xml"))
$1 ==> 2020-10-19T04:51:58.69531Z
jshell> new File("./pom.xml").lastModified()
$1 ==> 1603083118695
```
Would you update the comment like "Skip MacOS because HFS+ does not support
milliseconds for mtime." ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17310) Touch command with -c option is broken
[
https://issues.apache.org/jira/browse/HADOOP-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HADOOP-17310:
--
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Touch command with -c option is broken
> ---
>
> Key: HADOOP-17310
> URL: https://issues.apache.org/jira/browse/HADOOP-17310
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The touch command has an option -c :
> {noformat}
> touch
> Usage: hadoop fs -touch [-a] [-m] [-t TIMESTAMP] [-c] URI [URI ...]{noformat}
> Ref :
> [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#touch]
> But on using it, it gives error as :
> {noformat}
> -touch: Illegal option -c
> Usage: hadoop fs [generic options]{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17310) Touch command with -c option is broken
[
https://issues.apache.org/jira/browse/HADOOP-17310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216456#comment-17216456
]
Ayush Saxena commented on HADOOP-17310:
---
Merged PR to trunk!!!
> Touch command with -c option is broken
> ---
>
> Key: HADOOP-17310
> URL: https://issues.apache.org/jira/browse/HADOOP-17310
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The touch command has an option -c :
> {noformat}
> touch
> Usage: hadoop fs -touch [-a] [-m] [-t TIMESTAMP] [-c] URI [URI ...]{noformat}
> Ref :
> [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#touch]
> But on using it, it gives error as :
> {noformat}
> -touch: Illegal option -c
> Usage: hadoop fs [generic options]{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17310) Touch command with -c option is broken
[
https://issues.apache.org/jira/browse/HADOOP-17310?focusedWorklogId=502004=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502004
]
ASF GitHub Bot logged work on HADOOP-17310:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 05:25
Start Date: 19/Oct/20 05:25
Worklog Time Spent: 10m
Work Description: ayushtkn commented on pull request #2393:
URL: https://github.com/apache/hadoop/pull/2393#issuecomment-711596372
Thanx @ferhui for the review!!!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502004)
Time Spent: 1h (was: 50m)
> Touch command with -c option is broken
> ---
>
> Key: HADOOP-17310
> URL: https://issues.apache.org/jira/browse/HADOOP-17310
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The touch command has an option -c :
> {noformat}
> touch
> Usage: hadoop fs -touch [-a] [-m] [-t TIMESTAMP] [-c] URI [URI ...]{noformat}
> Ref :
> [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#touch]
> But on using it, it gives error as :
> {noformat}
> -touch: Illegal option -c
> Usage: hadoop fs [generic options]{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17306) RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
[
https://issues.apache.org/jira/browse/HADOOP-17306?focusedWorklogId=502003=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502003
]
ASF GitHub Bot logged work on HADOOP-17306:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 05:25
Start Date: 19/Oct/20 05:25
Worklog Time Spent: 10m
Work Description: aajisaka commented on a change in pull request #2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507473450
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
HFS+ does not support milliseconds for mtime, so we have to skip this.
In contrast, APFS supports nanoseconds and JDK-8177809 is fixed if the
filesystem is APFS.
```
➜ hadoop git:(trunk) uname -a
Darwin 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT
2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64
➜ hadoop git:(trunk) jshell
| Welcome to JShell -- Version 11.0.8
| For an introduction type: /help intro
jshell> Files.getLastModifiedTime(Paths.get("./pom.xml"))
$1 ==> 2020-10-19T04:51:58.69531Z
jshell> new File("./pom.xml").lastModified()
$1 ==> 1603083118695
```
Would you update the comment like "Skip MacOS because HFS+ does not support
milliseconds for mtime." ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502003)
Time Spent: 50m (was: 40m)
> RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
>
>
> Key: HADOOP-17306
> URL: https://issues.apache.org/jira/browse/HADOOP-17306
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> RawLocalFileSystem's FileStatus uses {{File.lastModified()}} api from JDK.
> This api looses milliseconds due to JDK bug.
> [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8177809]
> This bug fixed in JDK 10 b09 onwards and still exists in JDK 8 which is still
> being used in many productions.
> Apparently, {{Files.getLastModifiedTime()}} from java's nio package returns
> correct time.
> Use {{Files.getLastModifiedTime()}} instead of {{File.lastModified}} as
> workaround.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on pull request #2393: HADOOP-17310. Touch command with -c option is broken.
ayushtkn commented on pull request #2393: URL: https://github.com/apache/hadoop/pull/2393#issuecomment-711596372 Thanx @ferhui for the review!!! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2387: HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09
aajisaka commented on a change in pull request #2387:
URL: https://github.com/apache/hadoop/pull/2387#discussion_r507473450
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestRawLocalFileSystemContract.java
##
@@ -203,4 +203,27 @@ public void testPermission() throws Exception {
}
}
+ @Test
+ public void testMTimeAtime() throws IOException {
+RawLocalFileSystem.setUseDeprecatedFileStatus(true);
+try {
+ Path testDir = getTestBaseDir();
+ String testFilename = "testmtime";
+ Path path = new Path(testDir, testFilename);
+ Path file = new Path(path, "file");
+ fs.create(file);
+ long now = System.currentTimeMillis();
+ long mtime = (now % 1000 == 0) ? now + 1 : now;
+ long atime = (now % 1000 == 0) ? now + 2 : now;
+ fs.setTimes(file, mtime, atime);
+ FileStatus fileStatus = fs.getFileStatus(file);
+ if (!Shell.MAC) {
+//HADOOP-17306 ; JDK-8177809 not fixed in MacOs
+assertEquals(mtime, fileStatus.getModificationTime());
Review comment:
HFS+ does not support milliseconds for mtime, so we have to skip this.
In contrast, APFS supports nanoseconds and JDK-8177809 is fixed if the
filesystem is APFS.
```
➜ hadoop git:(trunk) uname -a
Darwin 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT
2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64
➜ hadoop git:(trunk) jshell
| Welcome to JShell -- Version 11.0.8
| For an introduction type: /help intro
jshell> Files.getLastModifiedTime(Paths.get("./pom.xml"))
$1 ==> 2020-10-19T04:51:58.69531Z
jshell> new File("./pom.xml").lastModified()
$1 ==> 1603083118695
```
Would you update the comment like "Skip MacOS because HFS+ does not support
milliseconds for mtime." ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn merged pull request #2393: HADOOP-17310. Touch command with -c option is broken.
ayushtkn merged pull request #2393: URL: https://github.com/apache/hadoop/pull/2393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17310) Touch command with -c option is broken
[
https://issues.apache.org/jira/browse/HADOOP-17310?focusedWorklogId=502002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502002
]
ASF GitHub Bot logged work on HADOOP-17310:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 05:24
Start Date: 19/Oct/20 05:24
Worklog Time Spent: 10m
Work Description: ayushtkn merged pull request #2393:
URL: https://github.com/apache/hadoop/pull/2393
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502002)
Time Spent: 50m (was: 40m)
> Touch command with -c option is broken
> ---
>
> Key: HADOOP-17310
> URL: https://issues.apache.org/jira/browse/HADOOP-17310
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The touch command has an option -c :
> {noformat}
> touch
> Usage: hadoop fs -touch [-a] [-m] [-t TIMESTAMP] [-c] URI [URI ...]{noformat}
> Ref :
> [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#touch]
> But on using it, it gives error as :
> {noformat}
> -touch: Illegal option -c
> Usage: hadoop fs [generic options]{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17311) ABFS: Logs should redact SAS signature
Sneha Vijayarajan created HADOOP-17311: -- Summary: ABFS: Logs should redact SAS signature Key: HADOOP-17311 URL: https://issues.apache.org/jira/browse/HADOOP-17311 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.0 Reporter: Sneha Vijayarajan Assignee: Sneha Vijayarajan Signature part of the SAS should be redacted for security purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15124) Slow FileSystem.Statistics counters implementation
[ https://issues.apache.org/jira/browse/HADOOP-15124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HADOOP-15124: --- Attachment: HADOOP-15124-branch-3.2.002.patch > Slow FileSystem.Statistics counters implementation > -- > > Key: HADOOP-15124 > URL: https://issues.apache.org/jira/browse/HADOOP-15124 > Project: Hadoop Common > Issue Type: Sub-task > Components: common >Affects Versions: 2.9.0, 2.8.3, 2.7.5, 3.0.0, 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: common, filesystem, fs, pull-request-available, > statistics > Attachments: HADOOP-15124-branch-3.2.001.patch, > HADOOP-15124-branch-3.2.002.patch, HADOOP-15124.001.patch > > Time Spent: 10m > Remaining Estimate: 0h > > While profiling 1TB TeraGen job on Hadoop 2.8.2 cluster (Google Dataproc, 2 > workers, GCS connector) I saw that FileSystem.Statistics code paths Wall time > is 5.58% and CPU time is 26.5% of total execution time. > After switching FileSystem.Statistics implementation to LongAdder, consumed > Wall time decreased to 0.006% and CPU time to 0.104% of total execution time. > Total job runtime decreased from 66 mins to 61 mins. > These results are not conclusive, because I didn't benchmark multiple times > to average results, but regardless of performance gains switching to > LongAdder simplifies code and reduces its complexity. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2394: HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client.
aajisaka commented on pull request #2394: URL: https://github.com/apache/hadoop/pull/2394#issuecomment-711478977 +1 pending Jenkins. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tasanuma opened a new pull request #2394: HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client.
tasanuma opened a new pull request #2394: URL: https://github.com/apache/hadoop/pull/2394 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17309) Javadoc warnings and errors are ignored in the precommit jobs
[ https://issues.apache.org/jira/browse/HADOOP-17309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-17309: --- Fix Version/s: 3.2.3 2.10.2 3.1.5 3.4.0 3.3.1 Resolution: Fixed Status: Resolved (was: Patch Available) Merged the PR into all the active branches. > Javadoc warnings and errors are ignored in the precommit jobs > - > > Key: HADOOP-17309 > URL: https://issues.apache.org/jira/browse/HADOOP-17309 > Project: Hadoop Common > Issue Type: Bug > Components: build, documentation >Affects Versions: 3.0.0, 2.10.0 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Fix For: 3.3.1, 3.4.0, 3.1.5, 2.10.2, 3.2.3 > > Time Spent: 40m > Remaining Estimate: 0h > > After changing the JDK from OracleJDK to the OpenJDK packaged in Ubuntu, > javadoc warnings and errors are not shown in the precommit Jenkins jobs. > In Ubuntu/Debian, doclint is not enabled by default: > - [https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1898116] > - > [https://github.com/Debian/openjdk-8/blob/master/debian/patches/disable-doclint-by-default.patch] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17310) Touch command with -c option is broken
[
https://issues.apache.org/jira/browse/HADOOP-17310?focusedWorklogId=501958=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501958
]
ASF GitHub Bot logged work on HADOOP-17310:
---
Author: ASF GitHub Bot
Created on: 19/Oct/20 02:10
Start Date: 19/Oct/20 02:10
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #2393:
URL: https://github.com/apache/hadoop/pull/2393#issuecomment-711470248
Good catch. +1
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501958)
Time Spent: 40m (was: 0.5h)
> Touch command with -c option is broken
> ---
>
> Key: HADOOP-17310
> URL: https://issues.apache.org/jira/browse/HADOOP-17310
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The touch command has an option -c :
> {noformat}
> touch
> Usage: hadoop fs -touch [-a] [-m] [-t TIMESTAMP] [-c] URI [URI ...]{noformat}
> Ref :
> [https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#touch]
> But on using it, it gives error as :
> {noformat}
> -touch: Illegal option -c
> Usage: hadoop fs [generic options]{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #2393: HADOOP-17310. Touch command with -c option is broken.
ferhui commented on pull request #2393: URL: https://github.com/apache/hadoop/pull/2393#issuecomment-711470248 Good catch. +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15814) Maven 3.3.3 unable to parse pom file
[
https://issues.apache.org/jira/browse/HADOOP-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216399#comment-17216399
]
Akira Ajisaka commented on HADOOP-15814:
Cherry-picked to branch-3.1 to backport HADOOP-17309 cleanly.
> Maven 3.3.3 unable to parse pom file
>
>
> Key: HADOOP-15814
> URL: https://issues.apache.org/jira/browse/HADOOP-15814
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.0, 3.3.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 3.2.0, 3.1.5
>
> Attachments: HADOOP-15814.001.patch
>
>
> Found via HDFS-13952.
> Reproducible on Maven 3.3.3, but not a problem for Maven 3.5.0 and above.
> I had to make the following change in order to compile. Looks like a problem
> after the ABFS merge.
> {code}
> diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml
> index cd38376..4c2c267 100644
> --- a/hadoop-project/pom.xml
> +++ b/hadoop-project/pom.xml
> @@ -1656,7 +1656,9 @@
>maven-javadoc-plugin
>${maven-javadoc-plugin.version}
>
> --Xmaxwarns 1
> +
> + -Xmaxwarns 1
> +
>
>
>
> {code}
> Otherwise it gives me this error:
> {quote}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar (module-javadocs) on
> project hadoop-project: Unable to parse configuration of mojo
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar for parameter
> additionalOptions: Cannot assign configuration entry 'additionalOptions' with
> value '-Xmaxwarns 1' of type java.lang.String to property of type
> java.lang.String[] -> [Help 1]
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15741) [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1
[ https://issues.apache.org/jira/browse/HADOOP-15741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216398#comment-17216398 ] Akira Ajisaka commented on HADOOP-15741: Modified the commit message and cherry-picked to branch-3.1 to backport HADOOP-17309 cleanly. > [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1 > --- > > Key: HADOOP-15741 > URL: https://issues.apache.org/jira/browse/HADOOP-15741 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Takanobu Asanuma >Priority: Minor > Fix For: 3.2.0, 3.1.5 > > Attachments: HADOOP-15741.1.patch, HADOOP-15741.2.patch > > > MJAVADOC-517 is fixed in 3.0.1. Let's upgrade the plugin to 3.0.1 or upper to > support Java 10. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15814) Maven 3.3.3 unable to parse pom file
[
https://issues.apache.org/jira/browse/HADOOP-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15814:
---
Fix Version/s: 3.1.5
> Maven 3.3.3 unable to parse pom file
>
>
> Key: HADOOP-15814
> URL: https://issues.apache.org/jira/browse/HADOOP-15814
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.0, 3.3.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 3.2.0, 3.3.0, 3.1.5
>
> Attachments: HADOOP-15814.001.patch
>
>
> Found via HDFS-13952.
> Reproducible on Maven 3.3.3, but not a problem for Maven 3.5.0 and above.
> I had to make the following change in order to compile. Looks like a problem
> after the ABFS merge.
> {code}
> diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml
> index cd38376..4c2c267 100644
> --- a/hadoop-project/pom.xml
> +++ b/hadoop-project/pom.xml
> @@ -1656,7 +1656,9 @@
>maven-javadoc-plugin
>${maven-javadoc-plugin.version}
>
> --Xmaxwarns 1
> +
> + -Xmaxwarns 1
> +
>
>
>
> {code}
> Otherwise it gives me this error:
> {quote}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar (module-javadocs) on
> project hadoop-project: Unable to parse configuration of mojo
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar for parameter
> additionalOptions: Cannot assign configuration entry 'additionalOptions' with
> value '-Xmaxwarns 1' of type java.lang.String to property of type
> java.lang.String[] -> [Help 1]
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15741) [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1
[ https://issues.apache.org/jira/browse/HADOOP-15741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-15741: --- Fix Version/s: 3.1.5 > [JDK10] Upgrade Maven Javadoc Plugin from 3.0.0-M1 to 3.0.1 > --- > > Key: HADOOP-15741 > URL: https://issues.apache.org/jira/browse/HADOOP-15741 > Project: Hadoop Common > Issue Type: Sub-task > Components: build >Reporter: Akira Ajisaka >Assignee: Takanobu Asanuma >Priority: Minor > Fix For: 3.2.0, 3.1.5 > > Attachments: HADOOP-15741.1.patch, HADOOP-15741.2.patch > > > MJAVADOC-517 is fixed in 3.0.1. Let's upgrade the plugin to 3.0.1 or upper to > support Java 10. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15814) Maven 3.3.3 unable to parse pom file
[
https://issues.apache.org/jira/browse/HADOOP-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HADOOP-15814:
---
Fix Version/s: (was: 3.3.0)
> Maven 3.3.3 unable to parse pom file
>
>
> Key: HADOOP-15814
> URL: https://issues.apache.org/jira/browse/HADOOP-15814
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.2.0, 3.3.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Fix For: 3.2.0, 3.1.5
>
> Attachments: HADOOP-15814.001.patch
>
>
> Found via HDFS-13952.
> Reproducible on Maven 3.3.3, but not a problem for Maven 3.5.0 and above.
> I had to make the following change in order to compile. Looks like a problem
> after the ABFS merge.
> {code}
> diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml
> index cd38376..4c2c267 100644
> --- a/hadoop-project/pom.xml
> +++ b/hadoop-project/pom.xml
> @@ -1656,7 +1656,9 @@
>maven-javadoc-plugin
>${maven-javadoc-plugin.version}
>
> --Xmaxwarns 1
> +
> + -Xmaxwarns 1
> +
>
>
>
> {code}
> Otherwise it gives me this error:
> {quote}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar (module-javadocs) on
> project hadoop-project: Unable to parse configuration of mojo
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:jar for parameter
> additionalOptions: Cannot assign configuration entry 'additionalOptions' with
> value '-Xmaxwarns 1' of type java.lang.String to property of type
> java.lang.String[] -> [Help 1]
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17309) Javadoc warnings and errors are ignored in the precommit jobs
[ https://issues.apache.org/jira/browse/HADOOP-17309?focusedWorklogId=501951=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501951 ] ASF GitHub Bot logged work on HADOOP-17309: --- Author: ASF GitHub Bot Created on: 19/Oct/20 01:48 Start Date: 19/Oct/20 01:48 Worklog Time Spent: 10m Work Description: aajisaka merged pull request #2391: URL: https://github.com/apache/hadoop/pull/2391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 501951) Time Spent: 0.5h (was: 20m) > Javadoc warnings and errors are ignored in the precommit jobs > - > > Key: HADOOP-17309 > URL: https://issues.apache.org/jira/browse/HADOOP-17309 > Project: Hadoop Common > Issue Type: Bug > Components: build, documentation >Affects Versions: 3.0.0, 2.10.0 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > After changing the JDK from OracleJDK to the OpenJDK packaged in Ubuntu, > javadoc warnings and errors are not shown in the precommit Jenkins jobs. > In Ubuntu/Debian, doclint is not enabled by default: > - [https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1898116] > - > [https://github.com/Debian/openjdk-8/blob/master/debian/patches/disable-doclint-by-default.patch] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17309) Javadoc warnings and errors are ignored in the precommit jobs
[ https://issues.apache.org/jira/browse/HADOOP-17309?focusedWorklogId=501952=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501952 ] ASF GitHub Bot logged work on HADOOP-17309: --- Author: ASF GitHub Bot Created on: 19/Oct/20 01:48 Start Date: 19/Oct/20 01:48 Worklog Time Spent: 10m Work Description: aajisaka commented on pull request #2391: URL: https://github.com/apache/hadoop/pull/2391#issuecomment-711465114 Thank you @tasanuma This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 501952) Time Spent: 40m (was: 0.5h) > Javadoc warnings and errors are ignored in the precommit jobs > - > > Key: HADOOP-17309 > URL: https://issues.apache.org/jira/browse/HADOOP-17309 > Project: Hadoop Common > Issue Type: Bug > Components: build, documentation >Affects Versions: 3.0.0, 2.10.0 >Reporter: Akira Ajisaka >Assignee: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > After changing the JDK from OracleJDK to the OpenJDK packaged in Ubuntu, > javadoc warnings and errors are not shown in the precommit Jenkins jobs. > In Ubuntu/Debian, doclint is not enabled by default: > - [https://bugs.launchpad.net/ubuntu/+source/openjdk-8/+bug/1898116] > - > [https://github.com/Debian/openjdk-8/blob/master/debian/patches/disable-doclint-by-default.patch] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2391: HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs
aajisaka commented on pull request #2391: URL: https://github.com/apache/hadoop/pull/2391#issuecomment-711465114 Thank you @tasanuma This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #2391: HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs
aajisaka merged pull request #2391: URL: https://github.com/apache/hadoop/pull/2391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on pull request #2390: YARN-10442. RM should make sure node label file highly available.
surendralilhore commented on pull request #2390: URL: https://github.com/apache/hadoop/pull/2390#issuecomment-711333021 @jiwq , Thanks for review. Updated the patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] surendralilhore commented on a change in pull request #2390: YARN-10442. RM should make sure node label file highly available.
surendralilhore commented on a change in pull request #2390:
URL: https://github.com/apache/hadoop/pull/2390#discussion_r507192427
##
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
##
@@ -4057,6 +4057,12 @@ public static boolean isAclEnabled(Configuration conf) {
public static final String FS_NODE_LABELS_STORE_ROOT_DIR = NODE_LABELS_PREFIX
+ "fs-store.root-dir";
+ /** Node label file replication */
+ public static final String FS_NODE_LABELS_FILE_REPLICATION =
+ NODE_LABELS_PREFIX + "file.replication";
+
Review comment:
Removed blank line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jiwq commented on a change in pull request #2390: YARN-10442. RM should make sure node label file highly available.
jiwq commented on a change in pull request #2390:
URL: https://github.com/apache/hadoop/pull/2390#discussion_r507177024
##
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
##
@@ -4057,6 +4057,12 @@ public static boolean isAclEnabled(Configuration conf) {
public static final String FS_NODE_LABELS_STORE_ROOT_DIR = NODE_LABELS_PREFIX
+ "fs-store.root-dir";
+ /** Node label file replication */
+ public static final String FS_NODE_LABELS_FILE_REPLICATION =
+ NODE_LABELS_PREFIX + "file.replication";
+
Review comment:
Remove this blank line, so that related configs can be group by.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2
[ https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HADOOP-17144: Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update Hadoop's lz4 to v1.9.2 > - > > Key: HADOOP-17144 > URL: https://issues.apache.org/jira/browse/HADOOP-17144 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Fix For: 3.4.0 > > Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, > HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch > > > Update hadoop's native lz4 to v1.9.2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17144) Update Hadoop's lz4 to v1.9.2
[ https://issues.apache.org/jira/browse/HADOOP-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216185#comment-17216185 ] Hemanth Boyina commented on HADOOP-17144: - thanks for the review [~iwasakims] committed to trunk > Update Hadoop's lz4 to v1.9.2 > - > > Key: HADOOP-17144 > URL: https://issues.apache.org/jira/browse/HADOOP-17144 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Hemanth Boyina >Assignee: Hemanth Boyina >Priority: Major > Attachments: HADOOP-17144.001.patch, HADOOP-17144.002.patch, > HADOOP-17144.003.patch, HADOOP-17144.004.patch, HADOOP-17144.005.patch > > > Update hadoop's native lz4 to v1.9.2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17265) ABFS: Support for Client Correlation ID
[
https://issues.apache.org/jira/browse/HADOOP-17265?focusedWorklogId=501868=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501868
]
ASF GitHub Bot logged work on HADOOP-17265:
---
Author: ASF GitHub Bot
Created on: 18/Oct/20 07:01
Start Date: 18/Oct/20 07:01
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2344:
URL: https://github.com/apache/hadoop/pull/2344#issuecomment-711126534
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 40m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 48s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 56s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 55s | | trunk passed |
| -0 :warning: | patch | 1m 12s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 54s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 0s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 1m 26s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/15/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 119m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azure.TestNativeAzureFileSystemConcurrency |
| | hadoop.fs.azure.TestNativeAzureFileSystemMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck |
| | hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemContractMocked |
| | hadoop.fs.azure.TestOutOfBandAzureBlobOperations |
| | hadoop.fs.azure.TestWasbFsck |
| | hadoop.fs.azure.TestBlobMetadata |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2344 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle
[GitHub] [hadoop] hadoop-yetus commented on pull request #2344: HADOOP-17265. ABFS: Support for Client Correlation ID
hadoop-yetus commented on pull request #2344:
URL: https://github.com/apache/hadoop/pull/2344#issuecomment-711126534
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 40m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 22s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 48s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 0m 56s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 55s | | trunk passed |
| -0 :warning: | patch | 1m 12s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 54s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 1m 0s | | the patch passed |
_ Other Tests _ |
| -1 :x: | unit | 1m 26s |
[/patch-unit-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/15/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt)
| hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 119m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.azure.TestNativeAzureFileSystemConcurrency |
| | hadoop.fs.azure.TestNativeAzureFileSystemMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck |
| | hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked |
| | hadoop.fs.azure.TestNativeAzureFileSystemContractMocked |
| | hadoop.fs.azure.TestOutOfBandAzureBlobOperations |
| | hadoop.fs.azure.TestWasbFsck |
| | hadoop.fs.azure.TestBlobMetadata |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2344/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2344 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle markdownlint |
| uname | Linux 80bc6600b319 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1e3a6efcef2 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions |
