[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914846#comment-16914846
]
Hudson commented on HDFS-14722:
---
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17182 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17182/])
HDFS-14722. RBF: GetMountPointStatus should return mountTable (ayushsaxena: rev
d2225c8ca8f9bdc5cef7266697518394d8763c88)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMountTable.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-006.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914794#comment-16914794
]
Ayush Saxena commented on HDFS-14722:
-
Committed to trunk.
Thanx [~xuzq_zander] for the contribution and [~elgoiri] for the review!!!
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-006.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914464#comment-16914464
]
Ayush Saxena commented on HDFS-14722:
-
Thanx [~xuzq_zander] for the patch. v006 LGTM +1
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-006.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913977#comment-16913977
]
Hadoop QA commented on HDFS-14722:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 52s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterFaultTolerant |
| | hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
| | hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HDFS-14722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978364/HDFS-14722-trunk-006.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 4002d46da1e1 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23
09:17:01 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f6af7d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27644/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/27644/testReport/ |
| Max. process+thread count | 1618 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913897#comment-16913897
]
xuzq commented on HDFS-14722:
-
Thanks [~elgoiri] for the comment. Fix the redundant variable in
[^HDFS-14722-trunk-006.patch] , please review it.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-006.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913894#comment-16913894
]
Íñigo Goiri commented on HDFS-14722:
This looks pretty much it.
A minor thing, the last assertEquals could just use the constant, no need to
have a variable and then check (it's a little verbose).
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913876#comment-16913876
]
xuzq commented on HDFS-14722:
-
Thanks [~ayushtkn] for the comment. Please review [^HDFS-14722-trunk-005.patch]
I have remove _routerContext.resetClient(null),_ because _dfsclient_ will use
new user to create new Connection in Client.java.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-005.patch, HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913573#comment-16913573
]
Ayush Saxena commented on HDFS-14722:
-
Thanx [~xuzq_zander] for the patch. Overall LGTM.
{code:java}
+ * Verify that the file/dir status with IOException in getMountPointDates.
{code}
This isn't in getMountPointDates. Shouldn't it be getMountPointStatus?
Secondly, {{routerContext.resetClient(null);}} is this required? I removed it
too, Still works fine. Give a check once
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913562#comment-16913562
]
Hadoop QA commented on HDFS-14722:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 22m 18s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
| | hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HDFS-14722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978276/HDFS-14722-trunk-004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5c41183a24cc 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 61b2df2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27629/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/27629/testReport/ |
| Max. process+thread count | 1559 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https:/
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913275#comment-16913275
]
xuzq commented on HDFS-14722:
-
Thanks [~ayushtkn], please review [^HDFS-14722-trunk-004.patch]
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-004.patch,
> HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911472#comment-16911472
]
Ayush Saxena commented on HDFS-14722:
-
Thanx [~xuzq_zander] for the patch. Do we need to add multiple entries for the
test, I guess adding one entry itself should work?
For the user part, can we use {{LambdaTestUtils.doAs(..)}}? rather than setting
unsettling login user?
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908001#comment-16908001
]
xuzq commented on HDFS-14722:
-
I create new Jira [HDFS-14739|https://issues.apache.org/jira/browse/HDFS-14739]
to discuss some other issues.
[~elgoiri] [~ayushtkn] please review [^HDFS-14722-trunk-003.patch] for this
issue, thanks.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906858#comment-16906858
]
Hadoop QA commented on HDFS-14722:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 3s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 39s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
| | hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
| | hadoop.hdfs.server.federation.router.TestRouterMountTableForBug |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HDFS-14722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977546/HDFS-14722-trunk-bug-discuss.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux d66e60d0ff57 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23
09:17:01 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e6d240d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27501/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/27501/testReport/ |
| Max. process+thread count | 1617 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoo
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906814#comment-16906814
]
xuzq commented on HDFS-14722:
-
I am so sorry, may be I didn't express it cleanly.
I attach [^HDFS-14722-trunk-bug-discuss.patch], please confirm these bug with
UT. Thanks
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch, HDFS-14722-trunk-bug-discuss.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906317#comment-16906317
]
Ayush Saxena commented on HDFS-14722:
-
I didn't catch the question only. :(
I understood the mount table structure, what is the response you are getting, I
couldn't catch.
bq. The owner of test1 in result should be mnt instead of test1？
why? That is mount entry owner. That could be test1 what is the problem?
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906300#comment-16906300
]
Hadoop QA commented on HDFS-14722:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
51s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 25s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 13s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.fs.contract.router.web.TestRouterWebHDFSContractDelete |
| | hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
| | hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HDFS-14722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977486/HDFS-14722-trunk-003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f9863e88dfc6 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0b507d2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27498/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/27498/testReport/ |
| Max. process+thread count | 1595 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rb
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906194#comment-16906194
]
xuzq commented on HDFS-14722:
-
Thanks [~ayushtkn] for the comment, and I attach a new patch, please review.
In addition to this problem, I think there is some bugs in *getListing*, please
check it.
*Bug one*:
||source||target namespace||destination||owner||group||permission||
|/mnt/test1|ns0|/mnt/test1|test1|test1|755|
|/mnt/test2|ns1|/mnt/test2|test2|test2|755|
The mount entry as above, the code as blew should return result?
And the result like: /mnt/test1 and /mnt/test2
{code:java}
routerProtocol.getListing("/mnt", HdfsFileStatus.EMPTY_NAME, false){code}
*Bug two:*
||source||target namespace||destination||owner||group||permission||
|/mnt|ns0|/mnt|mnt|mnt|755|
|/mnt/test1|ns1|/mnt/test1|mnt|mnt|755|
|/test1|ns2|/test1|test1|test1|755|
The mount entry as above, the code as blew.
The owner of *test1* should be mnt instead of test1？
*/test1* not the child of */mnt*, */mnt/test1* is the child of */mnt*.
{code:java}
routerProtocol.getListing("/mnt", HdfsFileStatus.EMPTY_NAME, false)
{code}
[~elgoiri] [~hexiaoqiao] [~ayushtkn] please confirm it.
If these are bugs, I will create new Jira to fix them.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch,
> HDFS-14722-trunk-003.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906017#comment-16906017
]
Ayush Saxena commented on HDFS-14722:
-
Thanx [~xuzq_zander] for the patch. We should have a separate UT for this,
similar to what we did in HDFS-14085 for several cases, rather than touching
the already existing ones. we can add one for this case. Give a check if you
can mock and generate IOE.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906011#comment-16906011
]
Hadoop QA commented on HDFS-14722:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 50s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs-rbf: The patch
generated 3 new + 5 unchanged - 0 fixed = 8 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 4s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 17s{color}
| {color:red} hadoop-hdfs-rbf in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 47s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterWithSecureStartup |
| | hadoop.hdfs.server.federation.security.TestRouterHttpDelegationToken |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HDFS-14722 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12977454/HDFS-14722-trunk-002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 9380a348d70b 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 454420e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/27493/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/27493/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://b
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905921#comment-16905921
]
He Xiaoqiao commented on HDFS-14722:
thanks [~xuzq_zander] for your contribution and trigger the Jenkins.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905915#comment-16905915
]
xuzq commented on HDFS-14722:
-
Thanks [~ayushtkn].
The method *getFileInfoAll* may be throw *AccessControlException*.
And I construct the AccessControlException in unit test. Please review the new
patch, thanks.
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch, HDFS-14722-trunk-002.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14722) RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException
[
https://issues.apache.org/jira/browse/HDFS-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905317#comment-16905317
]
Ayush Saxena commented on HDFS-14722:
-
Guess we should cover this with a Test too
> RBF: GetMountPointStatus should return mountTable information when
> getFileInfoAll throw IOException
> ---
>
> Key: HDFS-14722
> URL: https://issues.apache.org/jira/browse/HDFS-14722
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: xuzq
>Assignee: xuzq
>Priority: Major
> Attachments: HDFS-14722-trunk-001.patch
>
>
> When IOException in getFileInfoAll, we should return the mountTable
> informations instead of super information.
> Code like:
> {code:java}
> // RouterClientProtocol#getMountPointStatus
> try {
> String mName = name.startsWith("/") ? name : "/" + name;
> MountTableResolver mountTable = (MountTableResolver) subclusterResolver;
> MountTable entry = mountTable.getMountPoint(mName);
> if (entry != null) {
> RemoteMethod method = new RemoteMethod("getFileInfo",
> new Class[] {String.class}, new RemoteParam());
> HdfsFileStatus fInfo = getFileInfoAll(
> entry.getDestinations(), method, mountStatusTimeOut);
> if (fInfo != null) {
> permission = fInfo.getPermission();
> owner = fInfo.getOwner();
> group = fInfo.getGroup();
> childrenNum = fInfo.getChildrenNum();
> } else {
> permission = entry.getMode();
> owner = entry.getOwnerName();
> group = entry.getGroupName();
> }
> }
> } catch (IOException e) {
> LOG.error("Cannot get mount point: {}", e.getMessage());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
