[jira] [Commented] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
[ https://issues.apache.org/jira/browse/HDFS-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169738#comment-17169738 ] Leon Gao commented on HDFS-15509: - Attach a small fix > Set safemode should not fail if one of the namenode is down. > > > Key: HDFS-15509 > URL: https://issues.apache.org/jira/browse/HDFS-15509 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.0 >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > Attachments: HDFS-15509.patch > > > When the first namenode (let's say nn0) is down, set safemode command will > always fail unless users manually update the configuration. This is > distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169737#comment-17169737
]
Akira Ajisaka commented on HDFS-15508:
--
Attaching a patch to test PreCommit-HDFS-build job.
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: newbie
> Attachments: HDFS-15508.01.patch
>
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15508:
-
Attachment: HDFS-15508.01.patch
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: newbie
> Attachments: HDFS-15508.01.patch
>
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15508:
-
Status: Patch Available (was: Open)
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: newbie
> Attachments: HDFS-15508.01.patch
>
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
[ https://issues.apache.org/jira/browse/HDFS-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leon Gao updated HDFS-15509: Attachment: HDFS-15509.patch Status: Patch Available (was: Open) > Set safemode should not fail if one of the namenode is down. > > > Key: HDFS-15509 > URL: https://issues.apache.org/jira/browse/HDFS-15509 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.0 >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > Attachments: HDFS-15509.patch > > > When the first namenode (let's say nn0) is down, set safemode command will > always fail unless users manually update the configuration. This is > distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned HDFS-15508:
Assignee: Akira Ajisaka
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
[ https://issues.apache.org/jira/browse/HDFS-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leon Gao updated HDFS-15509: Status: Open (was: Patch Available) > Set safemode should not fail if one of the namenode is down. > > > Key: HDFS-15509 > URL: https://issues.apache.org/jira/browse/HDFS-15509 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.0 >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > > When the first namenode (let's say nn0) is down, set safemode command will > always fail unless users manually update the configuration. This is > distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
[ https://issues.apache.org/jira/browse/HDFS-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leon Gao updated HDFS-15509: Attachment: (was: HDFS-15509.patch) > Set safemode should not fail if one of the namenode is down. > > > Key: HDFS-15509 > URL: https://issues.apache.org/jira/browse/HDFS-15509 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.0 >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > > When the first namenode (let's say nn0) is down, set safemode command will > always fail unless users manually update the configuration. This is > distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
[ https://issues.apache.org/jira/browse/HDFS-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leon Gao updated HDFS-15509: Attachment: HDFS-15509.patch Status: Patch Available (was: Open) > Set safemode should not fail if one of the namenode is down. > > > Key: HDFS-15509 > URL: https://issues.apache.org/jira/browse/HDFS-15509 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.3.0 >Reporter: Leon Gao >Assignee: Leon Gao >Priority: Minor > > When the first namenode (let's say nn0) is down, set safemode command will > always fail unless users manually update the configuration. This is > distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15509) Set safemode should not fail if one of the namenode is down.
Leon Gao created HDFS-15509: --- Summary: Set safemode should not fail if one of the namenode is down. Key: HDFS-15509 URL: https://issues.apache.org/jira/browse/HDFS-15509 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Affects Versions: 3.3.0 Reporter: Leon Gao Assignee: Leon Gao When the first namenode (let's say nn0) is down, set safemode command will always fail unless users manually update the configuration. This is distracting when debugging issues. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15493) Update block map and name cache in parallel while loading fsimage.
[
https://issues.apache.org/jira/browse/HDFS-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169722#comment-17169722
]
Hadoop QA commented on HDFS-15493:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 4s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
57s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 39s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 44 unchanged - 0 fixed = 45 total (was 44) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 42s{color}
| {color:red} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}158m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| | hadoop.hdfs.server.datanode.TestBPOfferService |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/28/artifact/out/Dockerfile
|
| JIRA Issue | HDFS-15493 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13008908/HDFS-15493.005.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 8c7d76282d14 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Updated] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15508:
-
Target Version/s: 3.4.0
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Moved] (HDFS-15508) [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
[
https://issues.apache.org/jira/browse/HDFS-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka moved HADOOP-17176 to HDFS-15508:
---
Component/s: (was: documentation)
documentation
Key: HDFS-15508 (was: HADOOP-17176)
Target Version/s: (was: 3.4.0)
Project: Hadoop HDFS (was: Hadoop Common)
> [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
> -
>
> Key: HDFS-15508
> URL: https://issues.apache.org/jira/browse/HDFS-15508
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/security/token/package-info.java:21:
> error: reference not found
> [ERROR] * Implementations should extend {@link
> AbstractDelegationTokenSecretManager}.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a7dde76a4ba2942f60bf6230ec9ed6e1
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15506) [JDK 11] Fix javadoc errors in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15506:
-
Labels: newbie (was: )
> [JDK 11] Fix javadoc errors in hadoop-hdfs module
> -
>
> Key: HDFS-15506
> URL: https://issues.apache.org/jira/browse/HDFS-15506
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:43:
> error: self-closing element not allowed
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:682:
> error: malformed HTML
> [ERROR]* a NameNode per second. Values <= 0 disable throttling. This
> affects
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:1780:
> error: exception not thrown: java.io.FileNotFoundException
> [ERROR]* @throws FileNotFoundException
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java:176:
> error: @param name not found
> [ERROR]* @param mtime The snapshot creation time set by Time.now().
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:2187:
> error: exception not thrown: java.lang.Exception
> [ERROR]* @exception Exception if the filesystem does not exist.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15507) [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15507:
-
Description:
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientGSIContext.java:32:
error: self-closing element not allowed
[ERROR] *
[ERROR]^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
error: unexpected text
[ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
short, long,
[ERROR] ^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
error: reference not found
[ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until the
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
was:
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
error: unexpected text
[ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
short, long,
[ERROR] ^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
error: reference not found
[ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until the
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
> [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
>
>
> Key: HDFS-15507
> URL: https://issues.apache.org/jira/browse/HDFS-15507
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ClientGSIContext.java:32:
> error: self-closing element not allowed
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
> error: unexpected text
> [ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
> short, long,
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
> error: reference not found
> [ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until
> the
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15506) [JDK 11] Fix javadoc errors in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15506:
-
Description:
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:43:
error: self-closing element not allowed
[ERROR] *
[ERROR]^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:682:
error: malformed HTML
[ERROR]* a NameNode per second. Values <= 0 disable throttling. This affects
[ERROR]^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:1780:
error: exception not thrown: java.io.FileNotFoundException
[ERROR]* @throws FileNotFoundException
[ERROR] ^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java:176:
error: @param name not found
[ERROR]* @param mtime The snapshot creation time set by Time.now().
[ERROR] ^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:2187:
error: exception not thrown: java.lang.Exception
[ERROR]* @exception Exception if the filesystem does not exist.
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
was:
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java:54:
error: cannot find symbol
[ERROR] import org.apache.hadoop.hdfs.protocol.SnapshotStatus;
[ERROR] ^
[ERROR] symbol: class SnapshotStatus
[ERROR] location: package org.apache.hadoop.hdfs.protocol
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:102:
error: cannot find symbol
[ERROR] import org.apache.hadoop.hdfs.protocol.SnapshotStatus;
[ERROR] ^
[ERROR] symbol: class SnapshotStatus
[ERROR] location: package org.apache.hadoop.hdfs.protocol
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:7024:
error: cannot find symbol
[ERROR] public SnapshotStatus[] getSnapshotListing(String snapshotRoot)
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
> [JDK 11] Fix javadoc errors in hadoop-hdfs module
> -
>
> Key: HDFS-15506
> URL: https://issues.apache.org/jira/browse/HDFS-15506
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:43:
> error: self-closing element not allowed
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:682:
> error: malformed HTML
> [ERROR]* a NameNode per second. Values <= 0 disable throttling. This
> affects
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:1780:
> error: exception not thrown: java.io.FileNotFoundException
> [ERROR]* @throws FileNotFoundException
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java:176:
> error: @param name not found
> [ERROR]* @param mtime The snapshot creation time set by Time.now().
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:2187:
> error: exception not thrown: java.lang.Exception
> [ERROR]* @exception Exception if the filesystem does not exist.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn
[jira] [Updated] (HDFS-15507) [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15507:
-
Component/s: documentation
Labels: newbie (was: )
> [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
>
>
> Key: HDFS-15507
> URL: https://issues.apache.org/jira/browse/HDFS-15507
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Priority: Major
> Labels: newbie
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
> error: unexpected text
> [ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
> short, long,
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
> error: reference not found
> [ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until
> the
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15507) [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
[
https://issues.apache.org/jira/browse/HDFS-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-15507:
-
Summary: [JDK 11] Fix javadoc errors in hadoop-hdfs-client module (was:
[JDK 11] Fix javadoc errors in )
> [JDK 11] Fix javadoc errors in hadoop-hdfs-client module
>
>
> Key: HDFS-15507
> URL: https://issues.apache.org/jira/browse/HDFS-15507
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Priority: Major
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
> error: unexpected text
> [ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
> short, long,
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
> error: reference not found
> [ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until
> the
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15507) [JDK 11] Fix javadoc errors in
Akira Ajisaka created HDFS-15507:
Summary: [JDK 11] Fix javadoc errors in
Key: HDFS-15507
URL: https://issues.apache.org/jira/browse/HDFS-15507
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Akira Ajisaka
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1245:
error: unexpected text
[ERROR]* Same as {@link #create(String, FsPermission, EnumSet, boolean,
short, long,
[ERROR] ^
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java:161:
error: reference not found
[ERROR]* {@link HdfsConstants#LEASE_HARDLIMIT_PERIOD hard limit}. Until the
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/7ab1c48a9bd7a0fdb11fa82eb04874d5
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15506) [JDK 11] Fix javadoc errors in hadoop-hdfs module
Akira Ajisaka created HDFS-15506:
Summary: [JDK 11] Fix javadoc errors in hadoop-hdfs module
Key: HDFS-15506
URL: https://issues.apache.org/jira/browse/HDFS-15506
Project: Hadoop HDFS
Issue Type: Bug
Components: documentation
Reporter: Akira Ajisaka
{noformat}
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java:54:
error: cannot find symbol
[ERROR] import org.apache.hadoop.hdfs.protocol.SnapshotStatus;
[ERROR] ^
[ERROR] symbol: class SnapshotStatus
[ERROR] location: package org.apache.hadoop.hdfs.protocol
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:102:
error: cannot find symbol
[ERROR] import org.apache.hadoop.hdfs.protocol.SnapshotStatus;
[ERROR] ^
[ERROR] symbol: class SnapshotStatus
[ERROR] location: package org.apache.hadoop.hdfs.protocol
[ERROR]
/Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:7024:
error: cannot find symbol
[ERROR] public SnapshotStatus[] getSnapshotListing(String snapshotRoot)
[ERROR] ^
{noformat}
Full error log:
https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
How to reproduce the failure:
* Remove {{true}} from pom.xml
* Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15493) Update block map and name cache in parallel while loading fsimage.
[ https://issues.apache.org/jira/browse/HDFS-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169672#comment-17169672 ] Chengwei Wang commented on HDFS-15493: -- Submit a new patch [^HDFS-15493.005.patch] Removed the switch of this feature and simplified some related code. > Update block map and name cache in parallel while loading fsimage. > -- > > Key: HDFS-15493 > URL: https://issues.apache.org/jira/browse/HDFS-15493 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Chengwei Wang >Priority: Major > Attachments: HDFS-15493.001.patch, HDFS-15493.002.patch, > HDFS-15493.003.patch, HDFS-15493.004.patch, HDFS-15493.005.patch, > fsimage-loading.log > > > While loading INodeDirectorySection of fsimage, it will update name cache and > block map after added inode file to inode directory. It would reduce time > cost of fsimage loading to enable these steps run in parallel. > In our test case, with patch HDFS-13694 and HDFS-14617, the time cost to load > fsimage (220M files & 240M blocks) is 470s, with this patch , the time cost > reduc to 410s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15493) Update block map and name cache in parallel while loading fsimage.
[ https://issues.apache.org/jira/browse/HDFS-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chengwei Wang updated HDFS-15493: - Attachment: HDFS-15493.005.patch > Update block map and name cache in parallel while loading fsimage. > -- > > Key: HDFS-15493 > URL: https://issues.apache.org/jira/browse/HDFS-15493 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Chengwei Wang >Priority: Major > Attachments: HDFS-15493.001.patch, HDFS-15493.002.patch, > HDFS-15493.003.patch, HDFS-15493.004.patch, HDFS-15493.005.patch, > fsimage-loading.log > > > While loading INodeDirectorySection of fsimage, it will update name cache and > block map after added inode file to inode directory. It would reduce time > cost of fsimage loading to enable these steps run in parallel. > In our test case, with patch HDFS-13694 and HDFS-14617, the time cost to load > fsimage (220M files & 240M blocks) is 470s, with this patch , the time cost > reduc to 410s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15025) Applying NVDIMM storage media to HDFS
[ https://issues.apache.org/jira/browse/HDFS-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hadoop_hdfs_hw updated HDFS-15025: -- Attachment: HDFS-15025.006.patch Status: Patch Available (was: Open) > Applying NVDIMM storage media to HDFS > - > > Key: HDFS-15025 > URL: https://issues.apache.org/jira/browse/HDFS-15025 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, hdfs >Reporter: hadoop_hdfs_hw >Priority: Major > Attachments: Applying NVDIMM to HDFS.pdf, HDFS-15025.001.patch, > HDFS-15025.002.patch, HDFS-15025.003.patch, HDFS-15025.004.patch, > HDFS-15025.005.patch, HDFS-15025.006.patch, NVDIMM_patch(WIP).patch > > > The non-volatile memory NVDIMM is faster than SSD, it can be used > simultaneously with RAM, DISK, SSD. The data of HDFS stored directly on > NVDIMM can not only improves the response rate of HDFS, but also ensure the > reliability of the data. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15025) Applying NVDIMM storage media to HDFS
[ https://issues.apache.org/jira/browse/HDFS-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hadoop_hdfs_hw updated HDFS-15025: -- Status: Open (was: Patch Available) > Applying NVDIMM storage media to HDFS > - > > Key: HDFS-15025 > URL: https://issues.apache.org/jira/browse/HDFS-15025 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, hdfs >Reporter: hadoop_hdfs_hw >Priority: Major > Attachments: Applying NVDIMM to HDFS.pdf, HDFS-15025.001.patch, > HDFS-15025.002.patch, HDFS-15025.003.patch, HDFS-15025.004.patch, > HDFS-15025.005.patch, NVDIMM_patch(WIP).patch > > > The non-volatile memory NVDIMM is faster than SSD, it can be used > simultaneously with RAM, DISK, SSD. The data of HDFS stored directly on > NVDIMM can not only improves the response rate of HDFS, but also ensure the > reliability of the data. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15025) Applying NVDIMM storage media to HDFS
[ https://issues.apache.org/jira/browse/HDFS-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hadoop_hdfs_hw updated HDFS-15025: -- Attachment: (was: HDFS-15025.006.patch) > Applying NVDIMM storage media to HDFS > - > > Key: HDFS-15025 > URL: https://issues.apache.org/jira/browse/HDFS-15025 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, hdfs >Reporter: hadoop_hdfs_hw >Priority: Major > Attachments: Applying NVDIMM to HDFS.pdf, HDFS-15025.001.patch, > HDFS-15025.002.patch, HDFS-15025.003.patch, HDFS-15025.004.patch, > HDFS-15025.005.patch, NVDIMM_patch(WIP).patch > > > The non-volatile memory NVDIMM is faster than SSD, it can be used > simultaneously with RAM, DISK, SSD. The data of HDFS stored directly on > NVDIMM can not only improves the response rate of HDFS, but also ensure the > reliability of the data. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15493) Update block map and name cache in parallel while loading fsimage.
[
https://issues.apache.org/jira/browse/HDFS-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169665#comment-17169665
]
Chengwei Wang commented on HDFS-15493:
--
Hi [~sodonnell], thanks for your review.
{quote}I think we should create a new patch where the feature cannot be enabled
/ disabled - just have it always on, as I cannot think of a good reason someone
should turn it off, and it will make the code simpler if we just remove the
switch. What do you think?
{quote}
I agree with you. It's better to remove the switch of this feature. I will
remove it and submit a new patch soon.
> Update block map and name cache in parallel while loading fsimage.
> --
>
> Key: HDFS-15493
> URL: https://issues.apache.org/jira/browse/HDFS-15493
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Chengwei Wang
>Priority: Major
> Attachments: HDFS-15493.001.patch, HDFS-15493.002.patch,
> HDFS-15493.003.patch, HDFS-15493.004.patch, fsimage-loading.log
>
>
> While loading INodeDirectorySection of fsimage, it will update name cache and
> block map after added inode file to inode directory. It would reduce time
> cost of fsimage loading to enable these steps run in parallel.
> In our test case, with patch HDFS-13694 and HDFS-14617, the time cost to load
> fsimage (220M files & 240M blocks) is 470s, with this patch , the time cost
> reduc to 410s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15408) Failed execution caused by SocketTimeoutException
[
https://issues.apache.org/jira/browse/HDFS-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
echohlne updated HDFS-15408:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Failed execution caused by SocketTimeoutException
> -
>
> Key: HDFS-15408
> URL: https://issues.apache.org/jira/browse/HDFS-15408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
>Affects Versions: 3.1.1
>Reporter: echohlne
>Priority: Major
>
> When I execute command: hdfs fsck /
> in the hadoop cluster to check the health of the cluster, It always report
> an error execution failure like below:
> {code}
> Connecting to namenode via http://hadoop20:50070/fsck?ugi=hdfs=%2F
> Exception in thread "main" java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1587)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
> at org.apache.hadoop.hdfs.tools.DFSck.doWork(DFSck.java:359)
> at org.apache.hadoop.hdfs.tools.DFSck.access$000(DFSck.java:72)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:159)
> at org.apache.hadoop.hdfs.tools.DFSck$1.run(DFSck.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.hadoop.hdfs.tools.DFSck.run(DFSck.java:155)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSck.main(DFSck.java:402)
> {code}
> We try to solve this problem by adding a new parameter:
> {color:#de350b}*dfs.fsck.http.timeout.ms*{color} to control the
> connectionTimeout and the readTimeout if the HttpConnection in DFSck.java
> .Please check is it the right way to solve the problem? thanks a lot!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15498) Show snapshots deletion status in snapList cmd
[
https://issues.apache.org/jira/browse/HDFS-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169581#comment-17169581
]
Hudson commented on HDFS-15498:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18487 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18487/])
HDFS-15498. Show snapshots deletion status in snapList cmd. (#2181) (github:
rev d8a2df25ad2a145712910259f4d4079c302c2aef)
* (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestListSnapshot.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SnapshotStatus.java
> Show snapshots deletion status in snapList cmd
> --
>
> Key: HDFS-15498
> URL: https://issues.apache.org/jira/browse/HDFS-15498
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15498.000.patch
>
>
> HDFS-15488 adds a cmd to list all snapshots for a given snapshottable
> directory. A snapshot can be just marked as deleted with ordered deletion
> config set. This Jira aims to add deletion status to cmd output.
>
> SAMPLE OUTPUT:
> {noformat}
> sbanerjee-MBP15:hadoop-3.4.0-SNAPSHOT sbanerjee$ bin/hdfs lsSnapshottableDir
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:52 2 65536 /user
> sbanerjee-MBP15:hadoop-3.4.0-SNAPSHOT sbanerjee$ bin/hdfs lsSnapshot /user
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:52 1 ACTIVE
> /user/.snapshot/s1
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:51 0 DELETED
> /user/.snapshot/s20200727-115156.407{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15498) Show snapshots deletion status in snapList cmd
[
https://issues.apache.org/jira/browse/HDFS-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated HDFS-15498:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for the contribution [~shashikant] and [~hemanthboyina] for the reviews.
I have merged this change.
> Show snapshots deletion status in snapList cmd
> --
>
> Key: HDFS-15498
> URL: https://issues.apache.org/jira/browse/HDFS-15498
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15498.000.patch
>
>
> HDFS-15488 adds a cmd to list all snapshots for a given snapshottable
> directory. A snapshot can be just marked as deleted with ordered deletion
> config set. This Jira aims to add deletion status to cmd output.
>
> SAMPLE OUTPUT:
> {noformat}
> sbanerjee-MBP15:hadoop-3.4.0-SNAPSHOT sbanerjee$ bin/hdfs lsSnapshottableDir
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:52 2 65536 /user
> sbanerjee-MBP15:hadoop-3.4.0-SNAPSHOT sbanerjee$ bin/hdfs lsSnapshot /user
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:52 1 ACTIVE
> /user/.snapshot/s1
> drwxr-xr-x 0 sbanerjee supergroup 0 2020-07-27 11:51 0 DELETED
> /user/.snapshot/s20200727-115156.407{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15229) Truncate info should be logged at INFO level
[ https://issues.apache.org/jira/browse/HDFS-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169472#comment-17169472 ] Hudson commented on HDFS-15229: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18486 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18486/]) HDFS-15229. Truncate info should be logged at INFO level. Contributed by (hemanthboyina: rev 528a799a784c95cc702e3547a8a294f00743533b) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java > Truncate info should be logged at INFO level > - > > Key: HDFS-15229 > URL: https://issues.apache.org/jira/browse/HDFS-15229 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ravuri Sushma sree >Assignee: Ravuri Sushma sree >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-15229.001.patch, HDFS-15229.002.patch > > > In NN log and audit log, we can't find the truncate size. > Logs related to Truncate are captured at DEBUG Level and it is important that > NN should log the newLength of truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15229) Truncate info should be logged at INFO level
[ https://issues.apache.org/jira/browse/HDFS-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemanth Boyina updated HDFS-15229: -- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Truncate info should be logged at INFO level > - > > Key: HDFS-15229 > URL: https://issues.apache.org/jira/browse/HDFS-15229 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ravuri Sushma sree >Assignee: Ravuri Sushma sree >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-15229.001.patch, HDFS-15229.002.patch > > > In NN log and audit log, we can't find the truncate size. > Logs related to Truncate are captured at DEBUG Level and it is important that > NN should log the newLength of truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15229) Truncate info should be logged at INFO level
[ https://issues.apache.org/jira/browse/HDFS-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169470#comment-17169470 ] Hemanth Boyina commented on HDFS-15229: --- thank you [~Sushma_28] for the contribution committed to trunk > Truncate info should be logged at INFO level > - > > Key: HDFS-15229 > URL: https://issues.apache.org/jira/browse/HDFS-15229 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ravuri Sushma sree >Assignee: Ravuri Sushma sree >Priority: Major > Attachments: HDFS-15229.001.patch, HDFS-15229.002.patch > > > In NN log and audit log, we can't find the truncate size. > Logs related to Truncate are captured at DEBUG Level and it is important that > NN should log the newLength of truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15229) Truncate info should be logged at INFO level
[ https://issues.apache.org/jira/browse/HDFS-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169468#comment-17169468 ] Hemanth Boyina commented on HDFS-15229: --- +1 > Truncate info should be logged at INFO level > - > > Key: HDFS-15229 > URL: https://issues.apache.org/jira/browse/HDFS-15229 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ravuri Sushma sree >Assignee: Ravuri Sushma sree >Priority: Major > Attachments: HDFS-15229.001.patch, HDFS-15229.002.patch > > > In NN log and audit log, we can't find the truncate size. > Logs related to Truncate are captured at DEBUG Level and it is important that > NN should log the newLength of truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
