[jira] [Commented] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[ https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189294#comment-16189294 ] Xiaoyu Yao commented on HDFS-12568: --- +1 pending Jenkins. > Ozone: Cleanup the ozone-default.xml > > > Key: HDFS-12568 > URL: https://issues.apache.org/jira/browse/HDFS-12568 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Blocker > Labels: ozoneMerge > Attachments: HDFS-12568-HDFS-7240.001.patch, > HDFS-12568-HDFS-7240.002.patch, HDFS-12568-HDFS-7240.003.patch, > HDFS-12568-HDFS-7240.004.patch, HDFS-12568-HDFS-7240.005.patch > > > This JIRA proposes to clean up the ozone-default.xml before the merge. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189288#comment-16189288
]
Hadoop QA commented on HDFS-11968:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
30s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 3s{color} | {color:orange} root: The patch generated 3 new + 27 unchanged -
3 fixed = 30 total (was 30) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 21s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
12s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 34s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestLeaseRecoveryStriped |
| | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-11968 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890087/HDFS-11968.011.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux f5a37e49cc99 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / d4d2fd1 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
[jira] [Updated] (HDFS-12573) Divide the total block metrics into replica and ec
[ https://issues.apache.org/jira/browse/HDFS-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated HDFS-12573: Attachment: HDFS-12573.3.patch Thank you for your detailed review soon, [~manojg]! Uploaded the 3rd patch addressing your comments. > Divide the total block metrics into replica and ec > -- > > Key: HDFS-12573 > URL: https://issues.apache.org/jira/browse/HDFS-12573 > Project: Hadoop HDFS > Issue Type: Improvement > Components: erasure-coding, metrics, namenode >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma > Attachments: HDFS-12573.1.patch, HDFS-12573.2.patch, > HDFS-12573.3.patch > > > Following HDFS-10999, let's separate total blocks metrics. It would be useful > for administrators. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[
https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189264#comment-16189264
]
Hadoop QA commented on HDFS-12455:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 16s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 45s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
18s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
36s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 13s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
40s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestSafeModeWithStripedFile |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12455 |
| JIRA Patch URL |
[jira] [Updated] (HDFS-12513) Ozone: Create UI page to show Ozone configs by tags
[ https://issues.apache.org/jira/browse/HDFS-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12513: -- Status: Patch Available (was: Open) > Ozone: Create UI page to show Ozone configs by tags > --- > > Key: HDFS-12513 > URL: https://issues.apache.org/jira/browse/HDFS-12513 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: HDFS-7240 > > Attachments: HDFS-12513-HDFS-7240.001.patch, OzoneSettings.png > > > Create UI page to show Ozone configs by tags -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12513) Ozone: Create UI page to show Ozone configs by tags
[ https://issues.apache.org/jira/browse/HDFS-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12513: -- Attachment: HDFS-12513-HDFS-7240.001.patch > Ozone: Create UI page to show Ozone configs by tags > --- > > Key: HDFS-12513 > URL: https://issues.apache.org/jira/browse/HDFS-12513 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: HDFS-7240 > > Attachments: HDFS-12513-HDFS-7240.001.patch, OzoneSettings.png > > > Create UI page to show Ozone configs by tags -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[
https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189253#comment-16189253
]
Hadoop QA commented on HDFS-12273:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-10467 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
21s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} HDFS-10467 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 51s{color}
| {color:red} hadoop-hdfs-project_hadoop-hdfs generated 10 new + 384 unchanged
- 10 fixed = 394 total (was 394) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 11s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 13s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}139m 14s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.federation.metrics.TestFederationMetrics |
| | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.federation.router.TestNamenodeHeartbeat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12273 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890086/HDFS-12273-HDFS-10467-008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux 2ae278328e28 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-10467 / 39305af |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
[jira] [Commented] (HDFS-11858) JN httpServerUri should be set to hostname when using Default Http Address
[
https://issues.apache.org/jira/browse/HDFS-11858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189237#comment-16189237
]
Arpit Agarwal commented on HDFS-11858:
--
We should probably hold off on making this change. Currently the JN binds the
wildcard address by default. By restricting the bind address to a specific
hostname we may break some multi-homed deployments that depend on default
wildcard bind.
We could re-purpose this jira to document the fact that
GetJournalStateResponseProto.fromURL will be incorrect when the JN is using the
default http address. Also, I just noticed that the httpPort field in the
message is documented as deprecated. You could formalize it with the protobuf
{{deprecated}} option ([described
here|https://developers.google.com/protocol-buffers/docs/proto#options]).
> JN httpServerUri should be set to hostname when using Default Http Address
> --
>
> Key: HDFS-11858
> URL: https://issues.apache.org/jira/browse/HDFS-11858
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
> Attachments: HDFS-11858.001.patch
>
>
> Currently, when JN uses the default http address (0.0.0.0:8480), it sets the
> httpServerURI to 0.0.0.0:8480 as well. This value is passed as fromUrl
> address in GetEditLogManifestResponseProto and GetJournalStateResponseProto.
> When using the default http address, we should change the JN's httpServerURI
> to use the actual hostname.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[
https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-12568:
Attachment: HDFS-12568-HDFS-7240.005.patch
Cleanup some more, attached v5. Replaced /tmp with ${hadoop.tmp.dir}
> Ozone: Cleanup the ozone-default.xml
>
>
> Key: HDFS-12568
> URL: https://issues.apache.org/jira/browse/HDFS-12568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
>Priority: Blocker
> Labels: ozoneMerge
> Attachments: HDFS-12568-HDFS-7240.001.patch,
> HDFS-12568-HDFS-7240.002.patch, HDFS-12568-HDFS-7240.003.patch,
> HDFS-12568-HDFS-7240.004.patch, HDFS-12568-HDFS-7240.005.patch
>
>
> This JIRA proposes to clean up the ozone-default.xml before the merge.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12420) Add an option to disallow 'namenode format -force'
[
https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189228#comment-16189228
]
Arpit Agarwal commented on HDFS-12420:
--
UT failures are likely unrelated. I've retriggered Jenkins since there are so
many of them.
https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-HDFS-Build/21495/
> Add an option to disallow 'namenode format -force'
> --
>
> Key: HDFS-12420
> URL: https://issues.apache.org/jira/browse/HDFS-12420
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Ajay Kumar
>Assignee: Ajay Kumar
> Attachments: HDFS-12420.01.patch, HDFS-12420.02.patch,
> HDFS-12420.03.patch, HDFS-12420.04.patch, HDFS-12420.05.patch,
> HDFS-12420.06.patch, HDFS-12420.07.patch, HDFS-12420.08.patch,
> HDFS-12420.09.patch, HDFS-12420.10.patch, HDFS-12420.11.patch,
> HDFS-12420.12.patch
>
>
> Support for disabling NameNode format to avoid accidental formatting of
> Namenode in production cluster. If someone really wants to delete the
> complete fsImage, they can first delete the metadata dir and then run {code}
> hdfs namenode -format{code} manually.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188892#comment-16188892
]
Arpit Agarwal edited comment on HDFS-11968 at 10/3/17 3:46 AM:
---
Thanks [~msingh]. Your patch looks good to me. We can fix the behavior
introduced by HADOOP-13272 separately.
Two minor comments, but I am +1 basically.
# The {{instanceof}} check is unnecessary. You can just print the policy object
if it is non-null.
{code}
+ if ((policy != null) && (policy instanceof BlockStoragePolicy)) {
System.out.println("\t" + policy);
{code}
# The following error should go to stderr (via System.err.println)
{code}
System.out.println(getName() + " is not supported for filesystem "
+ fs.getScheme() + " on path " + path);
{code}
was (Author: arpitagarwal):
Thanks [~msingh]. Your patch looks good to me. We can fix the behavior
introduced by HDFS-13272 separately.
Two minor comments, but I am +1 basically.
# The {{instanceof}} check is unnecessary. You can just print the policy object
if it is non-null.
{code}
+ if ((policy != null) && (policy instanceof BlockStoragePolicy)) {
System.out.println("\t" + policy);
{code}
# The following error should go to stderr (via System.err.println)
{code}
System.out.println(getName() + " is not supported for filesystem "
+ fs.getScheme() + " on path " + path);
{code}
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch, HDFS-11968.011.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189221#comment-16189221
]
Arpit Agarwal edited comment on HDFS-11968 at 10/3/17 3:45 AM:
---
+1 for the v11 patch. Thanks [~msingh].
[~surendrasingh]/[~manojg], are you okay with the latest patch? I will hold off
committing for now in case you have additional comments.
was (Author: arpitagarwal):
+1 for the v11 patch. Thanks [~msingh].
[~surendrasingh]/[~manojg], are you okay with the latest patch? I will hold off
committing for now in you have additional comments.
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch, HDFS-11968.011.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189221#comment-16189221
]
Arpit Agarwal commented on HDFS-11968:
--
+1 for the v11 patch. Thanks [~msingh].
[~surendrasingh]/[~manojg], are you okay with the latest patch? I will hold off
committing for now in you have additional comments.
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch, HDFS-11968.011.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189218#comment-16189218 ] Xiaoyu Yao commented on HDFS-12455: --- Thanks [~ajayydv] for the update. +1 v5 patch pending Jenkins. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch, HDFS-12455.04.patch, HDFS-12455.05.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189189#comment-16189189
]
Mukul Kumar Singh commented on HDFS-11968:
--
Thanks for the review [~arpitagarwal], I have fixed the review comments in the
latest patch.
I have also raised HDFS-12581 to fix the issue with getAllStoragePolicies.
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch, HDFS-11968.011.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12581) ViewFileSystem#getAllStoragePolicies aggregates storage policies across NN using policyid
Mukul Kumar Singh created HDFS-12581:
Summary: ViewFileSystem#getAllStoragePolicies aggregates storage
policies across NN using policyid
Key: HDFS-12581
URL: https://issues.apache.org/jira/browse/HDFS-12581
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Reporter: Mukul Kumar Singh
Assignee: Mukul Kumar Singh
This jira was identified from review comments for HDFS-11968.
ViewFileSystem#getAllStoragePolicies aggregates storage policies across NN
using policyid,
However in a certain cluster,across nodes these storage policies can have
different meaning.
{code}
@Override
public Collection getAllStoragePolicies()
throws IOException {
Collection allPolicies = new HashSet<>();
for (FileSystem fs : getChildFileSystems()) {
try {
Collection policies =
fs.getAllStoragePolicies();
allPolicies.addAll(policies);
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[
https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-12568:
Attachment: HDFS-12568-HDFS-7240.004.patch
[~xyao] Thanks for the comments.
bq. Ozone-default.xml. Line 64: ozone->CBLOCK,PERFORMANCE?
Not able to locate this, skipping for now.
bq. Lin 126: can we change to dfs.cblock.container.size.gb to be consistent
with other Ozone keys related to size.
Fixed.
bq. Line 138/251: Should we use ${hadoop.tmp.dir}/cblockCacheDB?
Fixed.
bq. Line 327: should we add a CONTAINER tag for dfs.container.ipc.random.port?
Fixed.
bq. Line 337: should we add a RATIS tag for
dfs.container.ratis.datanode.storage.dir?
Fixed.
bq.Line 448: NIT: can you add xml comment to mark the beginning of the CLIENT
related keys?
Fixed, Added a comment for client settings for ozone client. There is still SCM
block client etc. which are used only by KSM.
> Ozone: Cleanup the ozone-default.xml
>
>
> Key: HDFS-12568
> URL: https://issues.apache.org/jira/browse/HDFS-12568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
>Priority: Blocker
> Labels: ozoneMerge
> Attachments: HDFS-12568-HDFS-7240.001.patch,
> HDFS-12568-HDFS-7240.002.patch, HDFS-12568-HDFS-7240.003.patch,
> HDFS-12568-HDFS-7240.004.patch
>
>
> This JIRA proposes to clean up the ozone-default.xml before the merge.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated HDFS-11968:
-
Attachment: HDFS-11968.011.patch
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch, HDFS-11968.011.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12273: --- Attachment: HDFS-12273-HDFS-10467-008.patch > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch, > HDFS-12273-HDFS-10467-007.patch, HDFS-12273-HDFS-10467-008.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12580) Rebasing HDFS-10467 after HDFS-12447
[
https://issues.apache.org/jira/browse/HDFS-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189161#comment-16189161
]
Íñigo Goiri commented on HDFS-12580:
As the branch is broken, I'm committing directly.
> Rebasing HDFS-10467 after HDFS-12447
>
>
> Key: HDFS-12580
> URL: https://issues.apache.org/jira/browse/HDFS-12580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
> Fix For: HDFS-10467
>
> Attachments: HDFS-12580-HDFS-10467.patch
>
>
> HDFS-12447 modified {{ClientProtocol#addErasureCodingPolicies}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12580) Rebasing HDFS-10467 after HDFS-12447
[
https://issues.apache.org/jira/browse/HDFS-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-12580:
---
Attachment: HDFS-12580-HDFS-10467.patch
> Rebasing HDFS-10467 after HDFS-12447
>
>
> Key: HDFS-12580
> URL: https://issues.apache.org/jira/browse/HDFS-12580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
> Fix For: HDFS-10467
>
> Attachments: HDFS-12580-HDFS-10467.patch
>
>
> HDFS-12447 modified {{ClientProtocol#addErasureCodingPolicies}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12580) Rebasing HDFS-10467 after HDFS-12447
[
https://issues.apache.org/jira/browse/HDFS-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-12580:
---
Attachment: (was: HDFS-12580.patch)
> Rebasing HDFS-10467 after HDFS-12447
>
>
> Key: HDFS-12580
> URL: https://issues.apache.org/jira/browse/HDFS-12580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
> Fix For: HDFS-10467
>
>
> HDFS-12447 modified {{ClientProtocol#addErasureCodingPolicies}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12580) Rebasing HDFS-10467 after HDFS-12447
[
https://issues.apache.org/jira/browse/HDFS-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-12580:
---
Attachment: HDFS-12580.patch
> Rebasing HDFS-10467 after HDFS-12447
>
>
> Key: HDFS-12580
> URL: https://issues.apache.org/jira/browse/HDFS-12580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
> Fix For: HDFS-10467
>
> Attachments: HDFS-12580.patch
>
>
> HDFS-12447 modified {{ClientProtocol#addErasureCodingPolicies}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12580) Rebasing HDFS-10467 after HDFS-12447
Íñigo Goiri created HDFS-12580:
--
Summary: Rebasing HDFS-10467 after HDFS-12447
Key: HDFS-12580
URL: https://issues.apache.org/jira/browse/HDFS-12580
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Íñigo Goiri
Assignee: Íñigo Goiri
HDFS-12447 modified {{ClientProtocol#addErasureCodingPolicies}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[
https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189148#comment-16189148
]
Hadoop QA commented on HDFS-12273:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-10467 Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 4m
35s{color} | {color:red} root in HDFS-10467 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
31s{color} | {color:red} hadoop-hdfs in HDFS-10467 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} HDFS-10467 passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
31s{color} | {color:red} hadoop-hdfs in HDFS-10467 failed. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
20s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red} hadoop-hdfs in HDFS-10467 failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} HDFS-10467 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 58s{color}
| {color:red} hadoop-hdfs-project_hadoop-hdfs generated 321 new + 73 unchanged
- 12 fixed = 394 total (was 85) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 1
unchanged - 2 fixed = 1 total (was 3) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}103m 50s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}132m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpc |
| | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.federation.metrics.TestFederationMetrics |
| | hadoop.hdfs.server.federation.router.TestNamenodeHeartbeat |
| | hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12273 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890065/HDFS-12273-HDFS-10467-007.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux b5ccc68fc57d 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HDFS-12573) Divide the total block metrics into replica and ec
[
https://issues.apache.org/jira/browse/HDFS-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189142#comment-16189142
]
Manoj Govindassamy commented on HDFS-12573:
---
Thanks for working on this [~tasanuma0829]. Looks good overall. +1 with few
comments below.
1. BlocksMap.java
{noformat}
void removeBlock(Block block) {
BlockInfo blockInfo = blocks.remove(block);
decrementBlockStat(block);
if (blockInfo == null)
return;
{noformat}
To be accurate, {{decrementBlockStat()}} need to be done after {{blockinfo ==
null}} check.
2.
bq. private void decrementBlockStat(Block block) {
We can have an assert to verify the count never goes below zero. I am ok even
if you don't do this for performance or other sanity reasons.
2. FSNameSystem.java
bq. @Metric({"TotalReplicatedBlocks", "Number of total replicated blocks"})
bq. @Metric({"TotalECBlockGroups", "Number of total erasure coded blocks"})
The comment can be "Total number of replicated blocks" and "Total number of
erasure coded block groups"
3. TestNameNodeMXBean.java
bq. verifyTotalBlocksMetrics(2L, namesystem.getTotalBlocks());
bq. void verifyTotalBlocksMetrics(long expectedBlocks, long actualBlocks)
There is also value in sending the expectedTotalReplicatedBlocks (1L) and
expectedTotalECBlockGroups (1L) number instead of the total expected blocks 2L.
In the verification method we can add up the individual expected blocks to get
the total expected blocks. This way we can make sure we are not assuming the
count from other block types. The test already has deletion of files and
re-verification of block counts which is very good. It would be even more
better, if we can add the expected block counts individually.
> Divide the total block metrics into replica and ec
> --
>
> Key: HDFS-12573
> URL: https://issues.apache.org/jira/browse/HDFS-12573
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, metrics, namenode
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
> Attachments: HDFS-12573.1.patch, HDFS-12573.2.patch
>
>
> Following HDFS-10999, let's separate total blocks metrics. It would be useful
> for administrators.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12573) Divide the total block metrics into replica and ec
[
https://issues.apache.org/jira/browse/HDFS-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189142#comment-16189142
]
Manoj Govindassamy edited comment on HDFS-12573 at 10/3/17 1:12 AM:
Thanks for working on this [~tasanuma0829]. Looks good overall. +1 with few
comments below.
1. BlocksMap.java
{noformat}
void removeBlock(Block block) {
BlockInfo blockInfo = blocks.remove(block);
decrementBlockStat(block);
if (blockInfo == null)
return;
{noformat}
To be accurate, {{decrementBlockStat()}} need to be done after {{blockinfo ==
null}} check.
2.
bq. private void decrementBlockStat(Block block) {
We can have an assert to verify the count never goes below zero. I am ok even
if you don't do this for performance or other sanity reasons.
3. FSNameSystem.java
bq. @Metric({"TotalReplicatedBlocks", "Number of total replicated blocks"})
bq. @Metric({"TotalECBlockGroups", "Number of total erasure coded blocks"})
The comment can be "Total number of replicated blocks" and "Total number of
erasure coded block groups"
4. TestNameNodeMXBean.java
bq. verifyTotalBlocksMetrics(2L, namesystem.getTotalBlocks());
bq. void verifyTotalBlocksMetrics(long expectedBlocks, long actualBlocks)
There is also value in sending the expectedTotalReplicatedBlocks (1L) and
expectedTotalECBlockGroups (1L) number instead of the total expected blocks 2L.
In the verification method we can add up the individual expected blocks to get
the total expected blocks. This way we can make sure we are not assuming the
count from other block types. The test already has deletion of files and
re-verification of block counts which is very good. It would be even more
better, if we can add the expected block counts individually.
was (Author: manojg):
Thanks for working on this [~tasanuma0829]. Looks good overall. +1 with few
comments below.
1. BlocksMap.java
{noformat}
void removeBlock(Block block) {
BlockInfo blockInfo = blocks.remove(block);
decrementBlockStat(block);
if (blockInfo == null)
return;
{noformat}
To be accurate, {{decrementBlockStat()}} need to be done after {{blockinfo ==
null}} check.
2.
bq. private void decrementBlockStat(Block block) {
We can have an assert to verify the count never goes below zero. I am ok even
if you don't do this for performance or other sanity reasons.
2. FSNameSystem.java
bq. @Metric({"TotalReplicatedBlocks", "Number of total replicated blocks"})
bq. @Metric({"TotalECBlockGroups", "Number of total erasure coded blocks"})
The comment can be "Total number of replicated blocks" and "Total number of
erasure coded block groups"
3. TestNameNodeMXBean.java
bq. verifyTotalBlocksMetrics(2L, namesystem.getTotalBlocks());
bq. void verifyTotalBlocksMetrics(long expectedBlocks, long actualBlocks)
There is also value in sending the expectedTotalReplicatedBlocks (1L) and
expectedTotalECBlockGroups (1L) number instead of the total expected blocks 2L.
In the verification method we can add up the individual expected blocks to get
the total expected blocks. This way we can make sure we are not assuming the
count from other block types. The test already has deletion of files and
re-verification of block counts which is very good. It would be even more
better, if we can add the expected block counts individually.
> Divide the total block metrics into replica and ec
> --
>
> Key: HDFS-12573
> URL: https://issues.apache.org/jira/browse/HDFS-12573
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, metrics, namenode
>Reporter: Takanobu Asanuma
>Assignee: Takanobu Asanuma
> Attachments: HDFS-12573.1.patch, HDFS-12573.2.patch
>
>
> Following HDFS-10999, let's separate total blocks metrics. It would be useful
> for administrators.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12455: -- Attachment: HDFS-12455.05.patch [~xyao], moved the shutdown to finally block in patch v5. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch, HDFS-12455.04.patch, HDFS-12455.05.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189136#comment-16189136 ] Ajay Kumar commented on HDFS-12455: --- test failures are unrelated. All 3 passes locally. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch, HDFS-12455.04.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12513) Ozone: Create UI page to show Ozone configs by tags
[ https://issues.apache.org/jira/browse/HDFS-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12513: -- Attachment: (was: HDFS-12513-HDFS-7240.001.patch) > Ozone: Create UI page to show Ozone configs by tags > --- > > Key: HDFS-12513 > URL: https://issues.apache.org/jira/browse/HDFS-12513 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: HDFS-7240 > > Attachments: OzoneSettings.png > > > Create UI page to show Ozone configs by tags -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12513) Ozone: Create UI page to show Ozone configs by tags
[ https://issues.apache.org/jira/browse/HDFS-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12513: -- Attachment: HDFS-12513-HDFS-7240.001.patch > Ozone: Create UI page to show Ozone configs by tags > --- > > Key: HDFS-12513 > URL: https://issues.apache.org/jira/browse/HDFS-12513 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: HDFS-7240 >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: HDFS-7240 > > Attachments: HDFS-12513-HDFS-7240.001.patch, OzoneSettings.png > > > Create UI page to show Ozone configs by tags -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189113#comment-16189113 ] Bharat Viswanadham commented on HDFS-12576: --- Thank You [~anu] for discussion and committing the patch. > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Fix For: 3.1.0 > > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[
https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189106#comment-16189106
]
Hadoop QA commented on HDFS-12568:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
49s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
34s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
11s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
16s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 19s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
16s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
4s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 57s{color} | {color:orange} root: The patch generated 4 new + 0 unchanged -
0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 50s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 11m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 10s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}210m 49s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}354m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestShellBasedUnixGroupsMapping |
| | hadoop.ha.TestZKFailoverController |
| | hadoop.security.TestKDiag |
| | hadoop.conf.TestCommonConfigurationFields |
| | hadoop.hdfs.TestPread |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.hdfs.qjournal.server.TestJournalNodeSync |
| |
[jira] [Comment Edited] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[
https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189076#comment-16189076
]
Xiaoyu Yao edited comment on HDFS-12455 at 10/3/17 12:09 AM:
-
Thanks [~ajayydv] for the update. Can we wrap the shutdown in the try-finally
so that if any exception happens during the test, we will always ensure the
cluster is shutdown properly?
{code}
MiniDFSCluster cluster = new MiniDFSCluster.Builder(new HdfsConfiguration())
.build();
try {
...
} finally {
cluster.shutdown();
}
{code}
We have seen random failures before caused by certain test that did not
shutdown cluster upon exception. The suggested change saves us time in tracing
and locating source of test failures like that.
was (Author: xyao):
Thanks [~ajayydv] for the update. +1 for v4 patch and will commit it shortly.
> WebHDFS - ListStatus query does not provide any information about a folder's
> "snapshot enabled" status
> --
>
> Key: HDFS-12455
> URL: https://issues.apache.org/jira/browse/HDFS-12455
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Ajay Kumar
>Assignee: Ajay Kumar
> Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch,
> HDFS-12455.03.patch, HDFS-12455.04.patch
>
>
> WebHDFS - ListStatus query does not provide any information about a folder's
> "snapshot enabled" status. Since "ListStatus" lists other attributes it will
> be good to include this attribute as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189076#comment-16189076 ] Xiaoyu Yao commented on HDFS-12455: --- Thanks [~ajayydv] for the update. +1 for v4 patch and will commit it shortly. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch, HDFS-12455.04.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[
https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189072#comment-16189072
]
Hadoop QA commented on HDFS-12455:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 44s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
43s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
22s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 90m 28s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
41s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}205m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12455 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890025/HDFS-12455.04.patch |
| Optional Tests |
[jira] [Commented] (HDFS-12273) Federation UI
[
https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189053#comment-16189053
]
Íñigo Goiri commented on HDFS-12273:
After rebase, the branch is broken because HDFS-12447 modified
{{ClientProtocol#addErasureCodingPolicies}}.
I can open a new JIRA for that but meanwhile, I already have it in version 007
of this patch.
> Federation UI
> -
>
> Key: HDFS-12273
> URL: https://issues.apache.org/jira/browse/HDFS-12273
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
> Fix For: HDFS-10467
>
> Attachments: federationUI-1.png, federationUI-2.png,
> federationUI-3.png, HDFS-12273-HDFS-10467-000.patch,
> HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch,
> HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch,
> HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch,
> HDFS-12273-HDFS-10467-007.patch
>
>
> Add the Web UI to the Router to expose the status of the federated cluster.
> It includes the federation metrics.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12578) TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7
[
https://issues.apache.org/jira/browse/HDFS-12578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Kumar reassigned HDFS-12578:
-
Assignee: Ajay Kumar
> TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7
>
>
> Key: HDFS-12578
> URL: https://issues.apache.org/jira/browse/HDFS-12578
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Reporter: Xiao Chen
>Assignee: Ajay Kumar
>Priority: Blocker
>
> It appears {{TestDeadDatanode#testNonDFSUsedONDeadNodeReReg}} is consistently
> failing in branch-2.7. We should investigate and fix it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-9003) ForkJoin thread pool leaks
[ https://issues.apache.org/jira/browse/HDFS-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189042#comment-16189042 ] Xiao Chen commented on HDFS-9003: - Backported this to branch-2.7 and branch-2.6. Thanks > ForkJoin thread pool leaks > -- > > Key: HDFS-9003 > URL: https://issues.apache.org/jira/browse/HDFS-9003 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.8.0, 3.0.0-alpha1, 2.6.6, 2.7.5 > > Attachments: HDFS-9003.patch > > > The pool is not shutdown, so each time a standby NN initializes the quota, > the pool leaks. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9003) ForkJoin thread pool leaks
[ https://issues.apache.org/jira/browse/HDFS-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9003: Fix Version/s: 2.7.5 2.6.6 > ForkJoin thread pool leaks > -- > > Key: HDFS-9003 > URL: https://issues.apache.org/jira/browse/HDFS-9003 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 2.8.0 >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.8.0, 3.0.0-alpha1, 2.6.6, 2.7.5 > > Attachments: HDFS-9003.patch > > > The pool is not shutdown, so each time a standby NN initializes the quota, > the pool leaks. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189025#comment-16189025 ] Hudson commented on HDFS-12576: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13007 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13007/]) HDFS-12576. JournalNodes are getting started, even though (aengineer: rev 015abcd8ce1d360830ee2960f1cdc743e09f1629) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/stop-dfs.sh > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Fix For: 3.1.0 > > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12577) Rename Router tooling
[
https://issues.apache.org/jira/browse/HDFS-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-12577:
---
Labels: RBF (was: )
> Rename Router tooling
> -
>
> Key: HDFS-12577
> URL: https://issues.apache.org/jira/browse/HDFS-12577
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Ajay Kumar
> Labels: RBF
> Fix For: HDFS-10467
>
>
> Currently the naming for Router Based Federation has a couple conflicts:
> * Both YARN and HDFS have a Router component which may cause issues for the
> PID file and JPS.
> * The tool to manage the mount table is called using {{hdfs federation}}.
> This may cause confusion with the regular HDFS federation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12273: --- Attachment: HDFS-12273-HDFS-10467-007.patch > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch, > HDFS-12273-HDFS-10467-007.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12577) Rename Router tooling
[
https://issues.apache.org/jira/browse/HDFS-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajay Kumar reassigned HDFS-12577:
-
Assignee: Ajay Kumar
> Rename Router tooling
> -
>
> Key: HDFS-12577
> URL: https://issues.apache.org/jira/browse/HDFS-12577
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
>Assignee: Ajay Kumar
> Fix For: HDFS-10467
>
>
> Currently the naming for Router Based Federation has a couple conflicts:
> * Both YARN and HDFS have a Router component which may cause issues for the
> PID file and JPS.
> * The tool to manage the mount table is called using {{hdfs federation}}.
> This may cause confusion with the regular HDFS federation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12579) JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url
[
https://issues.apache.org/jira/browse/HDFS-12579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189011#comment-16189011
]
Hanisha Koneru commented on HDFS-12579:
---
Thanks [~arpitagarwal] for pointing out this bug.
> JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to
> construct servlet Url
> --
>
> Key: HDFS-12579
> URL: https://issues.apache.org/jira/browse/HDFS-12579
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Hanisha Koneru
>Assignee: Hanisha Koneru
>
> Currently in JournalNodeSyncer, we construct the remote JN http server url
> using the JN host address and the http port that we get from the
> {{GetEditLogManifestResponseProto}}.
> {code}
> if (remoteJNproxy.httpServerUrl == null) {
> remoteJNproxy.httpServerUrl = getHttpServerURI("http",
> remoteJNproxy.jnAddr.getHostName(), response.getHttpPort());
> }
> {code}
> The correct way would be to get the http server url of the remote JN from the
> {{fromUrl}} field of the {{GetEditLogManifestResponseProto}}. This would take
> care of the http policy set on the remote JN as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12579) JournalNodeSyncer should use fromUrl field of EditLogManifestResponse to construct servlet Url
Hanisha Koneru created HDFS-12579:
-
Summary: JournalNodeSyncer should use fromUrl field of
EditLogManifestResponse to construct servlet Url
Key: HDFS-12579
URL: https://issues.apache.org/jira/browse/HDFS-12579
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Hanisha Koneru
Assignee: Hanisha Koneru
Currently in JournalNodeSyncer, we construct the remote JN http server url
using the JN host address and the http port that we get from the
{{GetEditLogManifestResponseProto}}.
{code}
if (remoteJNproxy.httpServerUrl == null) {
remoteJNproxy.httpServerUrl = getHttpServerURI("http",
remoteJNproxy.jnAddr.getHostName(), response.getHttpPort());
}
{code}
The correct way would be to get the http server url of the remote JN from the
{{fromUrl}} field of the {{GetEditLogManifestResponseProto}}. This would take
care of the http policy set on the remote JN as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12569) Unset EC policy logs empty payload in edit log
[
https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189007#comment-16189007
]
Hudson commented on HDFS-12569:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13006 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13006/])
HDFS-12569. Unset EC policy logs empty payload in edit log. (Lei (Eddy) (lei:
rev 27ffd43b6419c9ebe697536bcb6abb858ce791d2)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java
> Unset EC policy logs empty payload in edit log
> --
>
> Key: HDFS-12569
> URL: https://issues.apache.org/jira/browse/HDFS-12569
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HDFS-12569.00.patch, HDFS-12569.01.patch
>
>
> The edit log generated by {{hdfs ec -unsetPolicy}} generates an
> {{OP_REMOVE_XATTR}} entry in edit logs, but the payload like xattr namespace
> / name / vaue are missing:
> {code}
>
> OP_REMOVE_XATTR
>
> 420481
> /
> b098e758-9d7f-48b7-aa91-80ca52133b09
> 0
>
>
> {code}
> As a result, when Active NN restarts, or the Standby NN replay edits, this op
> has not effect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8865) Improve quota initialization performance
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8865: Resolution: Fixed Fix Version/s: 2.7.5 2.6.6 Status: Resolved (was: Patch Available) Thanks a lot [~yzhangal] for review. Pushed to branch-2.7 and branch-2.6. Compiled again before pushing. Also pushed the said series of jira to fix branch-2.6 test, and filed HDFS-12578 for branch-2.7. Thanks [~kihwal] [~nfraison.criteo] for contributing the patches, and [~jojochuang] for initial backport/review. > Improve quota initialization performance > > > Key: HDFS-8865 > URL: https://issues.apache.org/jira/browse/HDFS-8865 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.6.6, 2.7.5, 3.0.0-alpha1, 2.8.0 > > Attachments: HDFS-8865.branch-2.6.01.patch, > HDFS-8865_branch-2.6.patch, HDFS-8865.branch-2.6.patch, > HDFS-8865_branch-2.7.patch, HDFS-8865.branch-2.7.patch, HDFS-8865.patch, > HDFS-8865.v2.checkstyle.patch, HDFS-8865.v2.patch, HDFS-8865.v3.patch > > > After replaying edits, the whole file system tree is recursively scanned in > order to initialize the quota. For big name space, this can take a very long > time. Since this is done during namenode failover, it also affects failover > latency. > By using the Fork-Join framework, I was able to greatly reduce the > initialization time. The following is the test result using the fsimage from > one of the big name nodes we have. > || threads || seconds|| > | 1 (existing) | 55| > | 1 (fork-join) | 68 | > | 4 | 16 | > | 8 | 8 | > | 12 | 6 | > | 16 | 5 | > | 20 | 4 | -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[
https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188994#comment-16188994
]
Xiaoyu Yao commented on HDFS-12568:
---
The patch looks excellent to me. I just have few NITs in ozone-default.xml.
Ozone-default.xml
Line 64: ozone->CBLOCK,PERFORMANCE?
Lin 126: can we change to dfs.cblock.container.size.gb to be consistent with
other Ozone keys related to size.
Line 138/251: Should we use ${hadoop.tmp.dir}/cblockCacheDB?
Line 327: should we add a CONTAINER tag for dfs.container.ipc.random.port?
Line 337: should we add a RATIS tag for
dfs.container.ratis.datanode.storage.dir?
Line 448: NIT: can you add xml comment to mark the beginning of the CLIENT
related keys?
> Ozone: Cleanup the ozone-default.xml
>
>
> Key: HDFS-12568
> URL: https://issues.apache.org/jira/browse/HDFS-12568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Anu Engineer
>Assignee: Anu Engineer
>Priority: Blocker
> Labels: ozoneMerge
> Attachments: HDFS-12568-HDFS-7240.001.patch,
> HDFS-12568-HDFS-7240.002.patch, HDFS-12568-HDFS-7240.003.patch
>
>
> This JIRA proposes to clean up the ozone-default.xml before the merge.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-12576: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 Target Version/s: 3.1.0 Status: Resolved (was: Patch Available) [~bharatviswa] Thanks for the contribution and a very detailed explanation of what is happening. I have committed this patch to trunk. > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Fix For: 3.1.0 > > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8865) Improve quota initialization performance
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188983#comment-16188983 ] Yongjun Zhang commented on HDFS-8865: - Thanks [~xiaochen], +1 on the updated 2.6 patch (01). > Improve quota initialization performance > > > Key: HDFS-8865 > URL: https://issues.apache.org/jira/browse/HDFS-8865 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HDFS-8865.branch-2.6.01.patch, > HDFS-8865_branch-2.6.patch, HDFS-8865.branch-2.6.patch, > HDFS-8865_branch-2.7.patch, HDFS-8865.branch-2.7.patch, HDFS-8865.patch, > HDFS-8865.v2.checkstyle.patch, HDFS-8865.v2.patch, HDFS-8865.v3.patch > > > After replaying edits, the whole file system tree is recursively scanned in > order to initialize the quota. For big name space, this can take a very long > time. Since this is done during namenode failover, it also affects failover > latency. > By using the Fork-Join framework, I was able to greatly reduce the > initialization time. The following is the test result using the fsimage from > one of the big name nodes we have. > || threads || seconds|| > | 1 (existing) | 55| > | 1 (fork-join) | 68 | > | 4 | 16 | > | 8 | 8 | > | 12 | 6 | > | 16 | 5 | > | 20 | 4 | -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12569) Unset EC policy logs empty payload in edit log
[
https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-12569:
-
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Thanks for the review, [~andrew.wang]
Committed to {{trunk}} and {{branch-3.0}}
> Unset EC policy logs empty payload in edit log
> --
>
> Key: HDFS-12569
> URL: https://issues.apache.org/jira/browse/HDFS-12569
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HDFS-12569.00.patch, HDFS-12569.01.patch
>
>
> The edit log generated by {{hdfs ec -unsetPolicy}} generates an
> {{OP_REMOVE_XATTR}} entry in edit logs, but the payload like xattr namespace
> / name / vaue are missing:
> {code}
>
> OP_REMOVE_XATTR
>
> 420481
> /
> b098e758-9d7f-48b7-aa91-80ca52133b09
> 0
>
>
> {code}
> As a result, when Active NN restarts, or the Standby NN replay edits, this op
> has not effect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188975#comment-16188975 ] Íñigo Goiri commented on HDFS-12273: The errors are not related to this itself. It looks like the branch is out of sync with Yetus; I'll rebase HDFS-10467 and try again. > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188972#comment-16188972 ] Anu Engineer commented on HDFS-12576: - [~bharatviswa] Thanks for the very detailed off-line explanation. It looks like the older shell check is failing, this new check ensures that JNs are not started if the variable is indeed not set. To understand this change HDFS-12486 and HDFS-12375 patches are essential. The original sed based parsing was replaced with java code, but the return was an empty string with \n from the java code. The shell script checks for \n, but *IFS* strips it out. So, unfortunately, the shell check fails and Journal nodes are started even if the Journal nodes are not set. This new change fixes it via making sure that JOURNAL_NODE variable is not zero length instead of *'\n' check. This makes sure that JN is not started until we have a valid setting. +1, I will commit this shortly. > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[
https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188970#comment-16188970
]
Hadoop QA commented on HDFS-12273:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-10467 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
7s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
21s{color} | {color:green} HDFS-10467 passed {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 11m
37s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} HDFS-10467 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} HDFS-10467 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 46s{color}
| {color:red} hadoop-hdfs-project_hadoop-hdfs generated 10 new + 398 unchanged
- 10 fixed = 408 total (was 408) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 10m
16s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 26s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 22s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.server.federation.metrics.TestFederationMetrics |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.TestPread |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.TestClientProtocolForPipelineRecovery |
| | hadoop.hdfs.TestLeaseRecoveryStriped |
| Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12273 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890022/HDFS-12273-HDFS-10467-006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux 34864c3d3cd2 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-10467 / 2a75e9b |
|
[jira] [Commented] (HDFS-12569) Unset EC policy logs empty payload in edit log
[
https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188966#comment-16188966
]
Andrew Wang commented on HDFS-12569:
+1
> Unset EC policy logs empty payload in edit log
> --
>
> Key: HDFS-12569
> URL: https://issues.apache.org/jira/browse/HDFS-12569
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>Priority: Blocker
> Attachments: HDFS-12569.00.patch, HDFS-12569.01.patch
>
>
> The edit log generated by {{hdfs ec -unsetPolicy}} generates an
> {{OP_REMOVE_XATTR}} entry in edit logs, but the payload like xattr namespace
> / name / vaue are missing:
> {code}
>
> OP_REMOVE_XATTR
>
> 420481
> /
> b098e758-9d7f-48b7-aa91-80ca52133b09
> 0
>
>
> {code}
> As a result, when Active NN restarts, or the Standby NN replay edits, this op
> has not effect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12578) TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7
[
https://issues.apache.org/jira/browse/HDFS-12578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HDFS-12578:
-
Target Version/s: 2.7.5
> TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7
>
>
> Key: HDFS-12578
> URL: https://issues.apache.org/jira/browse/HDFS-12578
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
>Reporter: Xiao Chen
>Priority: Blocker
>
> It appears {{TestDeadDatanode#testNonDFSUsedONDeadNodeReReg}} is consistently
> failing in branch-2.7. We should investigate and fix it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12578) TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in branch-2.7
Xiao Chen created HDFS-12578:
Summary: TestDeadDatanode#testNonDFSUsedONDeadNodeReReg failing in
branch-2.7
Key: HDFS-12578
URL: https://issues.apache.org/jira/browse/HDFS-12578
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Reporter: Xiao Chen
Priority: Blocker
It appears {{TestDeadDatanode#testNonDFSUsedONDeadNodeReReg}} is consistently
failing in branch-2.7. We should investigate and fix it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8865) Improve quota initialization performance
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-8865: Attachment: HDFS-8865.branch-2.6.01.patch Thanks [~yzhangal], attached new 2.6 patch. > Improve quota initialization performance > > > Key: HDFS-8865 > URL: https://issues.apache.org/jira/browse/HDFS-8865 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HDFS-8865.branch-2.6.01.patch, > HDFS-8865_branch-2.6.patch, HDFS-8865.branch-2.6.patch, > HDFS-8865_branch-2.7.patch, HDFS-8865.branch-2.7.patch, HDFS-8865.patch, > HDFS-8865.v2.checkstyle.patch, HDFS-8865.v2.patch, HDFS-8865.v3.patch > > > After replaying edits, the whole file system tree is recursively scanned in > order to initialize the quota. For big name space, this can take a very long > time. Since this is done during namenode failover, it also affects failover > latency. > By using the Fork-Join framework, I was able to greatly reduce the > initialization time. The following is the test result using the fsimage from > one of the big name nodes we have. > || threads || seconds|| > | 1 (existing) | 55| > | 1 (fork-join) | 68 | > | 4 | 16 | > | 8 | 8 | > | 12 | 6 | > | 16 | 5 | > | 20 | 4 | -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12427) libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine
[
https://issues.apache.org/jira/browse/HDFS-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188945#comment-16188945
]
Hadoop QA commented on HDFS-12427:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} HDFS-8707 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
53s{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
15s{color} | {color:green} HDFS-8707 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
25s{color} | {color:green} HDFS-8707 passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 7m
30s{color} | {color:green} HDFS-8707 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
13s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 14m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
55s{color} | {color:green} the patch passed with JDK v1.7.0_151 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 14m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 8m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}312m 35s{color}
| {color:red} hadoop-hdfs-native-client in the patch failed with JDK
v1.7.0_151. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}444m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_151 Failed CTEST tests | test_hdfs_ext_hdfspp_test_shim_static |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:3117e2a |
| JIRA Issue | HDFS-12427 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889969/HDFS-12427.HDFS-8707.002.patch
|
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 14a0f06d8e99 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-8707 / 3f92e63 |
| Default Java | 1.7.0_151 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_144
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_151 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/21479/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client-jdk1.7.0_151.txt
|
| JDK v1.7.0_151 Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/21479/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/21479/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> libhdfs++: Prevent Requests from holding dangling pointer to RpcEngine
> --
>
> Key: HDFS-12427
> URL:
[jira] [Comment Edited] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188892#comment-16188892
]
Arpit Agarwal edited comment on HDFS-11968 at 10/2/17 9:40 PM:
---
Thanks [~msingh]. Your patch looks good to me. We can fix the behavior
introduced by HDFS-13272 separately.
Two minor comments, but I am +1 basically.
# The {{instanceof}} check is unnecessary. You can just print the policy object
if it is non-null.
{code}
+ if ((policy != null) && (policy instanceof BlockStoragePolicy)) {
System.out.println("\t" + policy);
{code}
# The following error should go to stderr (via System.err.println)
{code}
System.out.println(getName() + " is not supported for filesystem "
+ fs.getScheme() + " on path " + path);
{code}
was (Author: arpitagarwal):
Thanks [~msingh]. Your patch looks good to me. We can fix the behavior
introduced by HDFS-13272.
Two minor comments, but I am +1 basically.
# The {{instanceof}} check is unnecessary. You can just print the policy object
if it is non-null.
{code}
+ if ((policy != null) && (policy instanceof BlockStoragePolicy)) {
System.out.println("\t" + policy);
{code}
# The following error should go to stderr (via System.err.println)
{code}
System.out.println(getName() + " is not supported for filesystem "
+ fs.getScheme() + " on path " + path);
{code}
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11968) ViewFS: StoragePolicies commands fail with HDFS federation
[
https://issues.apache.org/jira/browse/HDFS-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188892#comment-16188892
]
Arpit Agarwal commented on HDFS-11968:
--
Thanks [~msingh]. Your patch looks good to me. We can fix the behavior
introduced by HDFS-13272.
Two minor comments, but I am +1 basically.
# The {{instanceof}} check is unnecessary. You can just print the policy object
if it is non-null.
{code}
+ if ((policy != null) && (policy instanceof BlockStoragePolicy)) {
System.out.println("\t" + policy);
{code}
# The following error should go to stderr (via System.err.println)
{code}
System.out.println(getName() + " is not supported for filesystem "
+ fs.getScheme() + " on path " + path);
{code}
> ViewFS: StoragePolicies commands fail with HDFS federation
> --
>
> Key: HDFS-11968
> URL: https://issues.apache.org/jira/browse/HDFS-11968
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
>Affects Versions: 2.7.1
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: HDFS-11968.001.patch, HDFS-11968.002.patch,
> HDFS-11968.003.patch, HDFS-11968.004.patch, HDFS-11968.005.patch,
> HDFS-11968.006.patch, HDFS-11968.007.patch, HDFS-11968.008.patch,
> HDFS-11968.009.patch, HDFS-11968.010.patch
>
>
> hdfs storagepolicies command fails with HDFS federation.
> For storage policies commands, a given user path should be resolved to a HDFS
> path and
> storage policy command should be applied onto the resolved HDFS path.
> {code}
> static DistributedFileSystem getDFS(Configuration conf)
> throws IOException {
> FileSystem fs = FileSystem.get(conf);
> if (!(fs instanceof DistributedFileSystem)) {
> throw new IllegalArgumentException("FileSystem " + fs.getUri() +
> " is not an HDFS file system");
> }
> return (DistributedFileSystem)fs;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1616#comment-1616 ] Anu Engineer commented on HDFS-12576: - Thank you for the explanation, it makes better sense now. However I do not agree with the fix, since does not even tell you why it is failing. I would suggest that we check this inside Journal node if this is indeed a bug and fail with a proper error so that user of the system is aware what needs to be done. Does that make sense? > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1611#comment-1611
]
Bharat Viswanadham edited comment on HDFS-12576 at 10/2/17 9:34 PM:
[~anu]
What is happening currently is
{code:java}
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
{code}
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
*Note: Internally getconf -journalnodes uses dfs.namenode.shared.edits.dir to
find list of journal nodes in the cluster.*
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
Pls let me know if it makes sense.
was (Author: bharatviswa):
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
*Note: Internally getconf -journalnodes uses dfs.namenode.shared.edits.dir to
find list of journal nodes in the cluster.*
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
Pls let me know if it makes sense.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1611#comment-1611
]
Bharat Viswanadham edited comment on HDFS-12576 at 10/2/17 9:33 PM:
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
*Note: Internally getconf -journalnodes uses dfs.namenode.shared.edits.dir to
find list of journal nodes in the cluster.*
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
Pls let me know if it makes sense.
was (Author: bharatviswa):
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
Pls let me know if it makes sense.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12577) Rename Router tooling
[
https://issues.apache.org/jira/browse/HDFS-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1610#comment-1610
]
Íñigo Goiri commented on HDFS-12577:
For the last few months, we've been getting feedback about the naming issues.
I'd like to use this JIRA to cover these.
I'd like to summarize the different comments:
* [~zhz] and [~shv] were concerned about distinguishing it from regular HDFS
federation.
* [~manojg] similarly thought the command line to manage the mount table could
get confusing as it currently used {{hdfs federation}}..
* [~aw] brought up the confusion of having two daemons (in YARN and HDFS)
called {{Router}}.
>From this feedback, I propose the following:
* Referring to this effort as "Router-Based Federation" or RBF.
* Create a class called {{HdfsRouter}} to start the Router, so the current main
from Router would move here. The command would still be {{hdfs router}} but it
would start {{HdfsRouter}} and avoid conflicts when calling {{jps}} etc.
* Rename the command I added for HDFS-10467 from {{hdfs federation}} to {{hdfs
routeradmin}}. Alternatively, I could merge both the Router start and the admin
into {{hdfs router}} and call {{HdfsRouter}} and {{RouterAdmin}} internally.
Thoughts?
> Rename Router tooling
> -
>
> Key: HDFS-12577
> URL: https://issues.apache.org/jira/browse/HDFS-12577
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
> Fix For: HDFS-10467
>
>
> Currently the naming for Router Based Federation has a couple conflicts:
> * Both YARN and HDFS have a Router component which may cause issues for the
> PID file and JPS.
> * The tool to manage the mount table is called using {{hdfs federation}}.
> This may cause confusion with the regular HDFS federation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1611#comment-1611
]
Bharat Viswanadham commented on HDFS-12576:
---
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1611#comment-1611
]
Bharat Viswanadham edited comment on HDFS-12576 at 10/2/17 9:32 PM:
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
Pls let me know if it makes sense.
was (Author: bharatviswa):
[~anu]
What is happening currently is
JOURNAL_NODES=$("${HADOOP_HDFS_HOME}/bin/hdfs" getconf -journalNodes 2>&-)
this will get an empty string with a new line, when
dfs.namenode.shared.edits.dir or dfs.namenode.shared.edits.dir.<>
is not set.
*Example Output:*
{code:java}
hdfs@4abb0327cd7f:/$ /hadoop/bin/hdfs getconf -journalnodes
2017-10-02 21:30:46,563 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
hdfs@4abb0327cd7f:/$
{code}
So, when nothing is set, the below check is passing ,
if [[ "{$JOURNAL_NODES-}" != $'\n' ]]; then
So, to avoid that checked the lenght of the returned journal nodes. If length
is greater than zero, there are journal nodes available, I need to start/stop
otherwise nothing is set, no need to start/stop journal nodes.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188872#comment-16188872 ] Sean Mackrory commented on HDFS-11096: -- Thanks, [~atm]! I have also not forgotten the comment about the SSH function likely breaking the existing usage. I'll check into that... > Support rolling upgrade between 2.x and 3.x > --- > > Key: HDFS-11096 > URL: https://issues.apache.org/jira/browse/HDFS-11096 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rolling upgrades >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Sean Mackrory >Priority: Blocker > Attachments: HDFS-11096.001.patch, HDFS-11096.002.patch, > HDFS-11096.003.patch, HDFS-11096.004.patch > > > trunk has a minimum software version of 3.0.0-alpha1. This means we can't > rolling upgrade between branch-2 and trunk. > This is a showstopper for large deployments. Unless there are very compelling > reasons to break compatibility, let's restore the ability to rolling upgrade > to 3.x releases. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188862#comment-16188862
]
Anu Engineer commented on HDFS-12576:
-
[~bharatviswa] I am sorry, but I don't see how this patch is adding the check
for a specific setting {{dfs.namenode.shared.edits.di}}, Could you please help
me understand this change better? Thanks in advance.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12577) Rename Router tooling
[
https://issues.apache.org/jira/browse/HDFS-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Íñigo Goiri updated HDFS-12577:
---
Description:
Currently the naming for Router Based Federation has a couple conflicts:
* Both YARN and HDFS have a Router component which may cause issues for the PID
file and JPS.
* The tool to manage the mount table is called using {{hdfs federation}}. This
may cause confusion with the regular HDFS federation.
> Rename Router tooling
> -
>
> Key: HDFS-12577
> URL: https://issues.apache.org/jira/browse/HDFS-12577
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
>Reporter: Íñigo Goiri
> Fix For: HDFS-10467
>
>
> Currently the naming for Router Based Federation has a couple conflicts:
> * Both YARN and HDFS have a Router component which may cause issues for the
> PID file and JPS.
> * The tool to manage the mount table is called using {{hdfs federation}}.
> This may cause confusion with the regular HDFS federation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12577) Rename Router tooling
Íñigo Goiri created HDFS-12577: -- Summary: Rename Router tooling Key: HDFS-12577 URL: https://issues.apache.org/jira/browse/HDFS-12577 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Íñigo Goiri -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12544) SnapshotDiff - support diff generation on any snapshot root descendant directory
[
https://issues.apache.org/jira/browse/HDFS-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188840#comment-16188840
]
Hadoop QA commented on HDFS-12544:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 25s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 54s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 466 unchanged - 5 fixed = 467 total (was 471) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 2s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 98m 44s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}149m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.ha.TestHASafeMode |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12544 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890001/HDFS-12544.02.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 727e6634dd6c 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5a68c2c |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/21484/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
[jira] [Commented] (HDFS-12569) Unset EC policy logs empty payload in edit log
[
https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188837#comment-16188837
]
Lei (Eddy) Xu commented on HDFS-12569:
--
Failed tests are not relevant, passed locally on my laptop.
> Unset EC policy logs empty payload in edit log
> --
>
> Key: HDFS-12569
> URL: https://issues.apache.org/jira/browse/HDFS-12569
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha4
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
>Priority: Blocker
> Attachments: HDFS-12569.00.patch, HDFS-12569.01.patch
>
>
> The edit log generated by {{hdfs ec -unsetPolicy}} generates an
> {{OP_REMOVE_XATTR}} entry in edit logs, but the payload like xattr namespace
> / name / vaue are missing:
> {code}
>
> OP_REMOVE_XATTR
>
> 420481
> /
> b098e758-9d7f-48b7-aa91-80ca52133b09
> 0
>
>
> {code}
> As a result, when Active NN restarts, or the Standby NN replay edits, this op
> has not effect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12543) Ozone : allow create key without specifying size
[ https://issues.apache.org/jira/browse/HDFS-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188788#comment-16188788 ] Chen Liang commented on HDFS-12543: --- The failed tests are unrelated, and passed in my local run. > Ozone : allow create key without specifying size > > > Key: HDFS-12543 > URL: https://issues.apache.org/jira/browse/HDFS-12543 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Chen Liang >Assignee: Chen Liang > Labels: ozoneMerge > Attachments: HDFS-12543-HDFS-7240.001.patch, > HDFS-12543-HDFS-7240.002.patch, HDFS-12543-HDFS-7240.003.patch, > HDFS-12543-HDFS-7240.004.patch, HDFS-12543-HDFS-7240.005.patch, > HDFS-12543-HDFS-7240.006.patch, HDFS-12543-HDFS-7240.007.patch, > HDFS-12543-HDFS-7240.008.patch > > > Currently when creating a key, it is required to specify the total size of > the key. This makes it inconvenient for the case where a key is created and > data keeps coming and being appended. This JIRA is remove the requirement of > specifying the size on key creation, and allows appending to the key > indefinitely. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12271) Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
[
https://issues.apache.org/jira/browse/HDFS-12271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188782#comment-16188782
]
Hudson commented on HDFS-12271:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13005 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13005/])
HDFS-12271. Incorrect statement in Downgrade section of HDFS Rolling (cliang:
rev 2c62ff7fd4392d55fe41573da297e2c0bc4f4c41)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md
> Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
> -
>
> Key: HDFS-12271
> URL: https://issues.apache.org/jira/browse/HDFS-12271
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Nandakumar
>Assignee: Nandakumar
>Priority: Minor
> Fix For: 3.1.0
>
> Attachments: HDFS-12271.000.patch
>
>
> In {{HDFS Rolling Upgrade}} document under {{Downgrade}} section, instruction
> given for {{Downgrade Active and Standby NNs}} has the following statement
> bq. Shutdown and upgrade NN1
> which should be
> bq. Shutdown and downgrade NN1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12569) Unset EC policy logs empty payload in edit log
[
https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188756#comment-16188756
]
Hadoop QA commented on HDFS-12569:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 36s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 48s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}135m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12569 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12889994/HDFS-12569.01.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 75d33fa1410f 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 563dcdf |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/21482/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/21482/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/21482/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Updated] (HDFS-12271) Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
[
https://issues.apache.org/jira/browse/HDFS-12271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang updated HDFS-12271:
--
Resolution: Fixed
Fix Version/s: 3.1.0
Target Version/s: 3.1.0
Status: Resolved (was: Patch Available)
> Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
> -
>
> Key: HDFS-12271
> URL: https://issues.apache.org/jira/browse/HDFS-12271
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Nandakumar
>Assignee: Nandakumar
>Priority: Minor
> Fix For: 3.1.0
>
> Attachments: HDFS-12271.000.patch
>
>
> In {{HDFS Rolling Upgrade}} document under {{Downgrade}} section, instruction
> given for {{Downgrade Active and Standby NNs}} has the following statement
> bq. Shutdown and upgrade NN1
> which should be
> bq. Shutdown and downgrade NN1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12271) Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
[
https://issues.apache.org/jira/browse/HDFS-12271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188751#comment-16188751
]
Chen Liang commented on HDFS-12271:
---
I've committed v000 patch to trunk, thanks [~nandakumar131] for the
contribution!
> Incorrect statement in Downgrade section of HDFS Rolling Upgrade document
> -
>
> Key: HDFS-12271
> URL: https://issues.apache.org/jira/browse/HDFS-12271
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Nandakumar
>Assignee: Nandakumar
>Priority: Minor
> Fix For: 3.1.0
>
> Attachments: HDFS-12271.000.patch
>
>
> In {{HDFS Rolling Upgrade}} document under {{Downgrade}} section, instruction
> given for {{Downgrade Active and Standby NNs}} has the following statement
> bq. Shutdown and upgrade NN1
> which should be
> bq. Shutdown and downgrade NN1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-12455: -- Attachment: HDFS-12455.04.patch [~xyao], thanks for review. Addressed it in patch v4. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch, HDFS-12455.04.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[
https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188742#comment-16188742
]
Hadoop QA commented on HDFS-12576:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
5s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 50s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
53s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12576 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890019/HDFS-12576.01.patch |
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux e23658568fbb 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5a68c2c |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/21485/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/21485/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> JournalNodes are getting started, even though dfs.namenode.shared.edits.dir
> is not configured
> -
>
> Key: HDFS-12576
> URL: https://issues.apache.org/jira/browse/HDFS-12576
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
> Attachments: HDFS-12576.01.patch
>
>
> dfs.namenode.shared.edits.dir is not configured, but still journal node is
> getting started.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188730#comment-16188730 ] Íñigo Goiri commented on HDFS-12273: Thanks [~giovanni.fumarola], I posted a couple fixes after testing the full HDFS-10467 in a dev cluster. Not sure what is causing the yetus errors, let's see if the new patch still has the issues. > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12110) libhdfs++: Rebase 8707 branch onto an up to date version of trunk
[
https://issues.apache.org/jira/browse/HDFS-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188715#comment-16188715
]
Hadoop QA commented on HDFS-12110:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HDFS-12110 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-12110 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12890021/HDFS-12110.diff |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/21486/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> libhdfs++: Rebase 8707 branch onto an up to date version of trunk
> -
>
> Key: HDFS-12110
> URL: https://issues.apache.org/jira/browse/HDFS-12110
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: James Clampffer
>Assignee: Deepak Majeti
> Attachments: HDFS-12110.diff, HDFS-12110.HDFS-8707.000.patch
>
>
> It's been way too long since this has been done and it's time to start
> knocking down blockers for merging into trunk. Can most likely just
> copy/paste the libhdfs++ directory into a newer version of master. Want to
> track it in a jira since it's likely to cause conflicts when pulling the
> updated branch for the first time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-12273: --- Attachment: HDFS-12273-HDFS-10467-006.patch > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch, HDFS-12273-HDFS-10467-006.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12110) libhdfs++: Rebase 8707 branch onto an up to date version of trunk
[ https://issues.apache.org/jira/browse/HDFS-12110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Majeti updated HDFS-12110: - Attachment: HDFS-12110.diff The docker related changes already exist on Hadoop master. My changes required to rebase are very minimal (Attached the diff). I will take a look at the dependencies. Thanks! > libhdfs++: Rebase 8707 branch onto an up to date version of trunk > - > > Key: HDFS-12110 > URL: https://issues.apache.org/jira/browse/HDFS-12110 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: James Clampffer >Assignee: Deepak Majeti > Attachments: HDFS-12110.diff, HDFS-12110.HDFS-8707.000.patch > > > It's been way too long since this has been done and it's time to start > knocking down blockers for merging into trunk. Can most likely just > copy/paste the libhdfs++ directory into a newer version of master. Want to > track it in a jira since it's likely to cause conflicts when pulling the > updated branch for the first time. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12528) Short-circuit reads getting disabled frequently in certain scenarios
[
https://issues.apache.org/jira/browse/HDFS-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge reassigned HDFS-12528:
-
Assignee: John Zhuge
> Short-circuit reads getting disabled frequently in certain scenarios
>
>
> Key: HDFS-12528
> URL: https://issues.apache.org/jira/browse/HDFS-12528
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client, performance
>Affects Versions: 2.6.0
>Reporter: Andre Araujo
>Assignee: John Zhuge
>
> We have scenarios where data ingestion makes use of the -appendToFile
> operation to add new data to existing HDFS files. In these situations, we're
> frequently running into the problem described below.
> We're using Impala to query the HDFS data with short-circuit reads (SCR)
> enabled. After each file read, Impala "unbuffer"'s the HDFS file to reduce
> the memory footprint. In some cases, though, Impala still keeps the HDFS file
> handle open for reuse.
> The "unbuffer" call, however, causes the file's current block reader to be
> closed, which makes the associated ShortCircuitReplica evictable from the
> ShortCircuitCache. When the cluster is under load, this means that the
> ShortCircuitReplica can be purged off the cache pretty fast, which closes the
> file descriptor to the underlying storage file.
> That means that when Impala re-reads the file it has to re-open the storage
> files associated with the ShortCircuitReplica's that were evicted from the
> cache. If there were no appends to those blocks, the re-open will succeed
> without problems. If one block was appended since the ShortCircuitReplica was
> created, the re-open will fail with the following error:
> {code}
> Meta file for BP-810388474-172.31.113.69-1499543341726:blk_1074012183_273087
> not found
> {code}
> This error is handled as an "unknown response" by the BlockReaderFactory [1],
> which disables short-circuit reads for 10 minutes [2] for the client.
> These 10 minutes without SCR can have a big performance impact for the client
> operations. In this particular case ("Meta file not found") it would suffice
> to return null without disabling SCR. This particular block read would fall
> back to the normal, non-short-circuited, path and other SCR requests would
> continue to work as expected.
> It might also be interesting to be able to control how long SCR is disabled
> for in the "unknown response" case. 10 minutes seems a bit to long and not
> being able to change that is a problem.
> [1]
> https://github.com/apache/hadoop/blob/f67237cbe7bc48a1b9088e990800b37529f1db2a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderFactory.java#L646
> [2]
> https://github.com/apache/hadoop/blob/f67237cbe7bc48a1b9088e990800b37529f1db2a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/DomainSocketFactory.java#L97
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8865) Improve quota initialization performance
[ https://issues.apache.org/jira/browse/HDFS-8865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188666#comment-16188666 ] Yongjun Zhang commented on HDFS-8865: - Thanks [~xiaochen] for working on this. +1 on the new revs for 2.7. Question on 2.6: Good to add the test to the 2.6 new patch. There is only one new test for this jira in 2.7 and beyond. Though it helps testing by including additional tests, should the other tests come with backporting different jiras (HDFS-10843 etc)? Minor: when uploading new patches for the same branch in the future, suggest to include a version string. > Improve quota initialization performance > > > Key: HDFS-8865 > URL: https://issues.apache.org/jira/browse/HDFS-8865 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Kihwal Lee >Assignee: Kihwal Lee > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: HDFS-8865_branch-2.6.patch, HDFS-8865.branch-2.6.patch, > HDFS-8865_branch-2.7.patch, HDFS-8865.branch-2.7.patch, HDFS-8865.patch, > HDFS-8865.v2.checkstyle.patch, HDFS-8865.v2.patch, HDFS-8865.v3.patch > > > After replaying edits, the whole file system tree is recursively scanned in > order to initialize the quota. For big name space, this can take a very long > time. Since this is done during namenode failover, it also affects failover > latency. > By using the Fork-Join framework, I was able to greatly reduce the > initialization time. The following is the test result using the fsimage from > one of the big name nodes we have. > || threads || seconds|| > | 1 (existing) | 55| > | 1 (fork-join) | 68 | > | 4 | 16 | > | 8 | 8 | > | 12 | 6 | > | 16 | 5 | > | 20 | 4 | -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12576: -- Status: Patch Available (was: In Progress) > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12576: -- Attachment: HDFS-12576.01.patch > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > Attachments: HDFS-12576.01.patch > > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12455) WebHDFS - ListStatus query does not provide any information about a folder's "snapshot enabled" status
[ https://issues.apache.org/jira/browse/HDFS-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188657#comment-16188657 ] Xiaoyu Yao commented on HDFS-12455: --- Thanks [~ajayydv] for working on this. The patch looks pretty good to me.The new unit test has one issue below. +1 after that being fixed. TestDistributedFileSystem.java Line 1274: the test spins up a MiniDFSCluster and it has to be shutdown once done. > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status > -- > > Key: HDFS-12455 > URL: https://issues.apache.org/jira/browse/HDFS-12455 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Attachments: HDFS-12455.01.patch, HDFS-12455.02.patch, > HDFS-12455.03.patch > > > WebHDFS - ListStatus query does not provide any information about a folder's > "snapshot enabled" status. Since "ListStatus" lists other attributes it will > be good to include this attribute as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12568) Ozone: Cleanup the ozone-default.xml
[ https://issues.apache.org/jira/browse/HDFS-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188645#comment-16188645 ] Ajay Kumar commented on HDFS-12568: --- [~anu], thanks for patch. I applied it locally. LGTM. +1 (non-binding) > Ozone: Cleanup the ozone-default.xml > > > Key: HDFS-12568 > URL: https://issues.apache.org/jira/browse/HDFS-12568 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Blocker > Labels: ozoneMerge > Attachments: HDFS-12568-HDFS-7240.001.patch, > HDFS-12568-HDFS-7240.002.patch, HDFS-12568-HDFS-7240.003.patch > > > This JIRA proposes to clean up the ozone-default.xml before the merge. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12576) JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12576: -- Summary: JournalNodes are getting started, even though dfs.namenode.shared.edits.dir is not configured (was: JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is not configured) > JournalNodes are getting started, even though dfs.namenode.shared.edits.dir > is not configured > - > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12576) JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is not configured
Bharat Viswanadham created HDFS-12576: - Summary: JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is not configured Key: HDFS-12576 URL: https://issues.apache.org/jira/browse/HDFS-12576 Project: Hadoop HDFS Issue Type: Bug Reporter: Bharat Viswanadham dfs.namenode.shared.edits.dir is not configured, but still journal node is getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-12576) JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12576 started by Bharat Viswanadham. - > JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is > not configured > > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-12576) JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is not configured
[ https://issues.apache.org/jira/browse/HDFS-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham reassigned HDFS-12576: - Assignee: Bharat Viswanadham > JournalNodes are getting started, even hough dfs.namenode.shared.edits.dir is > not configured > > > Key: HDFS-12576 > URL: https://issues.apache.org/jira/browse/HDFS-12576 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham > > dfs.namenode.shared.edits.dir is not configured, but still journal node is > getting started. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12060) Ozone: OzoneClient: Add list calls in RpcClient
[
https://issues.apache.org/jira/browse/HDFS-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao updated HDFS-12060:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: HDFS-7240
Status: Resolved (was: Patch Available)
Thanks [~nandakumar131] for the contribution. I've commit the patch to the
feature branch.
> Ozone: OzoneClient: Add list calls in RpcClient
> ---
>
> Key: HDFS-12060
> URL: https://issues.apache.org/jira/browse/HDFS-12060
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Nandakumar
>Assignee: Nandakumar
> Labels: ozoneMerge
> Fix For: HDFS-7240
>
> Attachments: HDFS-12060-HDFS-7240.000.patch,
> HDFS-12060-HDFS-7240.001.patch, HDFS-12060-HDFS-7240.002.patch,
> HDFS-12060-HDFS-7240.003.patch
>
>
> Support for {{listVolumes}}, {{listBuckets}}, {{listKeys}} in {{RpcClient}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12060) Ozone: OzoneClient: Add list calls in RpcClient
[
https://issues.apache.org/jira/browse/HDFS-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188614#comment-16188614
]
Xiaoyu Yao commented on HDFS-12060:
---
Thanks [~nandakumar131] for the update. Patch v3 looks good to me, +1. I will
commit it shortly.
> Ozone: OzoneClient: Add list calls in RpcClient
> ---
>
> Key: HDFS-12060
> URL: https://issues.apache.org/jira/browse/HDFS-12060
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Nandakumar
>Assignee: Nandakumar
> Labels: ozoneMerge
> Attachments: HDFS-12060-HDFS-7240.000.patch,
> HDFS-12060-HDFS-7240.001.patch, HDFS-12060-HDFS-7240.002.patch,
> HDFS-12060-HDFS-7240.003.patch
>
>
> Support for {{listVolumes}}, {{listBuckets}}, {{listKeys}} in {{RpcClient}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12552) Use slf4j instead of log4j in FSNamesystem
[ https://issues.apache.org/jira/browse/HDFS-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188607#comment-16188607 ] Hudson commented on HDFS-12552: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13003 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13003/]) HDFS-12552. Use slf4j instead of log4j in FSNamesystem. Contributed by (cliang: rev 5a68c2cef31031522f2532064965f482017dfefd) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestPersistBlocks.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadWhileWriting.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileConcurrentReader.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreationClient.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemLock.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLeaseRecovery2.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatus.java > Use slf4j instead of log4j in FSNamesystem > -- > > Key: HDFS-12552 > URL: https://issues.apache.org/jira/browse/HDFS-12552 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: 3.1.0 > > Attachments: HDFS-12552.01.patch, HDFS-12552.02.patch > > > FileNamesystem is still using log4j dependencies. We should move those to > slf4j, as most of the methods using log4j are deprecated. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12552) Use slf4j instead of log4j in FSNamesystem
[ https://issues.apache.org/jira/browse/HDFS-12552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188602#comment-16188602 ] Ajay Kumar commented on HDFS-12552: --- [~vagarychen], thanks for review and commit. > Use slf4j instead of log4j in FSNamesystem > -- > > Key: HDFS-12552 > URL: https://issues.apache.org/jira/browse/HDFS-12552 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ajay Kumar >Assignee: Ajay Kumar > Fix For: 3.1.0 > > Attachments: HDFS-12552.01.patch, HDFS-12552.02.patch > > > FileNamesystem is still using log4j dependencies. We should move those to > slf4j, as most of the methods using log4j are deprecated. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12273) Federation UI
[ https://issues.apache.org/jira/browse/HDFS-12273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188603#comment-16188603 ] Giovanni Matteo Fumarola commented on HDFS-12273: - Thanks [~elgoiri] for the patch. LGTM. Can you fix the yetus warnings? > Federation UI > - > > Key: HDFS-12273 > URL: https://issues.apache.org/jira/browse/HDFS-12273 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: fs >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri > Fix For: HDFS-10467 > > Attachments: federationUI-1.png, federationUI-2.png, > federationUI-3.png, HDFS-12273-HDFS-10467-000.patch, > HDFS-12273-HDFS-10467-001.patch, HDFS-12273-HDFS-10467-002.patch, > HDFS-12273-HDFS-10467-003.patch, HDFS-12273-HDFS-10467-004.patch, > HDFS-12273-HDFS-10467-005.patch > > > Add the Web UI to the Router to expose the status of the federated cluster. > It includes the federation metrics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
