[jira] [Commented] (HDFS-12086) Ozone: Add the unit test for KSMMetrics
[
https://issues.apache.org/jira/browse/HDFS-12086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074283#comment-16074283
]
Hadoop QA commented on HDFS-12086:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
16s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 44s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 53s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}144m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM |
| | hadoop.ozone.TestOzoneConfigurationFields |
| | hadoop.cblock.TestCBlockReadWrite |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.ozone.scm.node.TestNodeManager |
| | hadoop.ozone.container.ozoneimpl.TestOzoneContainerRatis |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.ozone.container.ozoneimpl.TestRatisManager |
| Timed out junit tests | org.apache.hadoop.cblock.TestLocalBlockCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12086 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875684/HDFS-12086-HDFS-7240.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e9864e98c45b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / b23b016 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20157/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20157/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20157/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20157/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
[jira] [Commented] (HDFS-12087) The error message is not friendly when set a path with the policy not enabled
[
https://issues.apache.org/jira/browse/HDFS-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074250#comment-16074250
]
Hadoop QA commented on HDFS-12087:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
42s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 28s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12087 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875686/HDFS-12087.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux edaf3b225800 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20156/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20156/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20156/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20156/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> The error message is not friendly when set a path with the policy not enabled
> -
>
> Key: HDFS-12087
> URL:
[jira] [Updated] (HDFS-11975) Provide a system-default EC policy
[
https://issues.apache.org/jira/browse/HDFS-11975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
luhuichun updated HDFS-11975:
-
Attachment: HDFS-11975-005.patch
update with some minor change
> Provide a system-default EC policy
> --
>
> Key: HDFS-11975
> URL: https://issues.apache.org/jira/browse/HDFS-11975
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: luhuichun
> Attachments: HDFS-11975-001.patch, HDFS-11975-002.patch,
> HDFS-11975-003.patch, HDFS-11975-004.patch, HDFS-11975-005.patch
>
>
> From the usability point of view, it'd be nice to be able to specify a
> system-wide EC policy, i.e., in {{hdfs-site.xml}}. For most of users / admin
> / downstream projects, it is not necessary to know the tradeoffs of the EC
> policy, considering that it requires the knowledge of EC, the actual physical
> topology of the clusters, and many other factors (i.e., network, cluster size
> and etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12087) The error message is not friendly when set a path with the policy not enabled
[
https://issues.apache.org/jira/browse/HDFS-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lufei updated HDFS-12087:
-
Target Version/s: 3.0.0-beta1 (was: 3.0.0-alpha4)
Status: Patch Available (was: Open)
> The error message is not friendly when set a path with the policy not enabled
> -
>
> Key: HDFS-12087
> URL: https://issues.apache.org/jira/browse/HDFS-12087
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.0.0-alpha3
>Reporter: lufei
>Assignee: lufei
> Attachments: HDFS-12087.001.patch
>
>
> First user add a policy by -addPolicies command but not enabled, then user
> set a path with this policy. The error message displayed as below:
> {color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
> enabled erasure coding policies: []. The set of enabled erasure coding
> policies can be configured at 'dfs.namenode.ec.policies.enabled'.{color}
> The policy 'XOR-2-1-128k' is added by user but not be enabled.The error
> message is not promot user to enable the policy first.I think the error
> message may be better as below:
> {color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
> enabled erasure coding policies: []. The set of enabled erasure coding
> policies can be configured at 'dfs.namenode.ec.policies.enabled' or enable
> the policy by '-enablePolicy' EC command before.{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12087) The error message is not friendly when set a path with the policy not enabled
[
https://issues.apache.org/jira/browse/HDFS-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lufei updated HDFS-12087:
-
Attachment: HDFS-12087.001.patch
> The error message is not friendly when set a path with the policy not enabled
> -
>
> Key: HDFS-12087
> URL: https://issues.apache.org/jira/browse/HDFS-12087
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.0.0-alpha3
>Reporter: lufei
>Assignee: lufei
> Attachments: HDFS-12087.001.patch
>
>
> First user add a policy by -addPolicies command but not enabled, then user
> set a path with this policy. The error message displayed as below:
> {color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
> enabled erasure coding policies: []. The set of enabled erasure coding
> policies can be configured at 'dfs.namenode.ec.policies.enabled'.{color}
> The policy 'XOR-2-1-128k' is added by user but not be enabled.The error
> message is not promot user to enable the policy first.I think the error
> message may be better as below:
> {color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
> enabled erasure coding policies: []. The set of enabled erasure coding
> policies can be configured at 'dfs.namenode.ec.policies.enabled' or enable
> the policy by '-enablePolicy' EC command before.{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12086) Ozone: Add the unit test for KSMMetrics
[ https://issues.apache.org/jira/browse/HDFS-12086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-12086: - Status: Patch Available (was: Open) > Ozone: Add the unit test for KSMMetrics > --- > > Key: HDFS-12086 > URL: https://issues.apache.org/jira/browse/HDFS-12086 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: test >Affects Versions: HDFS-7240 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: HDFS-12086-HDFS-7240.001.patch > > > Currently the unit test for KSMMetrics is missing. And some metrics name is > inconsistent with that in documentation: > * numVolumeModifies should be numVolumeUpdates > * numBucketModifies should be numBucketUpdates -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12086) Ozone: Add the unit test for KSMMetrics
[
https://issues.apache.org/jira/browse/HDFS-12086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12086:
-
Attachment: HDFS-12086-HDFS-7240.001.patch
Attach the initial patch. Some notes will help you review:
* New added each request type metrics for volume, bucket and key ops.
* Update the inconsistent metric names.
* Add the new test class for testing all the existing metrics in
{{KSMMetrics.java}}
Any comments is welcomed.
> Ozone: Add the unit test for KSMMetrics
> ---
>
> Key: HDFS-12086
> URL: https://issues.apache.org/jira/browse/HDFS-12086
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: test
>Affects Versions: HDFS-7240
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-12086-HDFS-7240.001.patch
>
>
> Currently the unit test for KSMMetrics is missing. And some metrics name is
> inconsistent with that in documentation:
> * numVolumeModifies should be numVolumeUpdates
> * numBucketModifies should be numBucketUpdates
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12087) The error message is not friendly when set a path with the policy not enabled
lufei created HDFS-12087:
Summary: The error message is not friendly when set a path with
the policy not enabled
Key: HDFS-12087
URL: https://issues.apache.org/jira/browse/HDFS-12087
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs
Affects Versions: 3.0.0-alpha3
Reporter: lufei
Assignee: lufei
First user add a policy by -addPolicies command but not enabled, then user set
a path with this policy. The error message displayed as below:
{color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
enabled erasure coding policies: []. The set of enabled erasure coding policies
can be configured at 'dfs.namenode.ec.policies.enabled'.{color}
The policy 'XOR-2-1-128k' is added by user but not be enabled.The error message
is not promot user to enable the policy first.I think the error message may be
better as below:
{color:#707070}RemoteException: Policy 'XOR-2-1-128k' does not match any
enabled erasure coding policies: []. The set of enabled erasure coding policies
can be configured at 'dfs.namenode.ec.policies.enabled' or enable the policy by
'-enablePolicy' EC command before.{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12086) Ozone: Add the unit test for KSMMetrics
Yiqun Lin created HDFS-12086: Summary: Ozone: Add the unit test for KSMMetrics Key: HDFS-12086 URL: https://issues.apache.org/jira/browse/HDFS-12086 Project: Hadoop HDFS Issue Type: Sub-task Components: test Affects Versions: HDFS-7240 Reporter: Yiqun Lin Assignee: Yiqun Lin Currently the unit test for KSMMetrics is missing. And some metrics name is inconsistent with that in documentation: * numVolumeModifies should be numVolumeUpdates * numBucketModifies should be numBucketUpdates -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074182#comment-16074182
]
Hadoop QA commented on HDFS-12082:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
39s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 41 unchanged - 2 fixed = 41 total (was 43) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 47s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12082 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875680/HDFS-12082.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 00247461dc88 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20155/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20155/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20155/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20155/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
>
[jira] [Updated] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12082:
---
Attachment: HDFS-12082.002.patch
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch, HDFS-12082.002.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12085) Reconfigure namenode interval fails if the interval was set with time unit
[
https://issues.apache.org/jira/browse/HDFS-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073993#comment-16073993
]
Hadoop QA commented on HDFS-12085:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
13s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
54s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
40s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 27s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestMaintenanceState |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| Timed out junit tests |
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12085 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875659/HDFS-12085.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 247e005c0c86 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20154/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20154/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output |
[jira] [Commented] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073866#comment-16073866
]
Hadoop QA commented on HDFS-12082:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
39s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 33s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 41 unchanged - 2 fixed = 42 total (was 43) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m 1s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12082 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875639/HDFS-12082.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f3b258454320 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20153/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20153/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20153/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20153/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20153/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Updated] (HDFS-12085) Reconfigure namenode interval fails if the interval was set with time unit
[
https://issues.apache.org/jira/browse/HDFS-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12085:
---
Status: Patch Available (was: Open)
> Reconfigure namenode interval fails if the interval was set with time unit
> --
>
> Key: HDFS-12085
> URL: https://issues.apache.org/jira/browse/HDFS-12085
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, tools
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Critical
> Attachments: HDFS-12085.001.patch
>
>
> It fails when I set duration with time unit, e.g 5s, error
> {noformat}
> Reconfiguring status for node [localhost:8111]: started at Tue Jul 04
> 08:14:18 PDT 2017 and finished at Tue Jul 04 08:14:18 PDT 2017.
> FAILED: Change property dfs.heartbeat.interval
> From: "3s"
> To: "5s"
> Error: For input string: "5s".
> {noformat}
> time unit support was added via HDFS-9847.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12085) Reconfigure namenode interval fails if the interval was set with time unit
[
https://issues.apache.org/jira/browse/HDFS-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12085:
---
Attachment: HDFS-12085.001.patch
> Reconfigure namenode interval fails if the interval was set with time unit
> --
>
> Key: HDFS-12085
> URL: https://issues.apache.org/jira/browse/HDFS-12085
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, tools
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Critical
> Attachments: HDFS-12085.001.patch
>
>
> It fails when I set duration with time unit, e.g 5s, error
> {noformat}
> Reconfiguring status for node [localhost:8111]: started at Tue Jul 04
> 08:14:18 PDT 2017 and finished at Tue Jul 04 08:14:18 PDT 2017.
> FAILED: Change property dfs.heartbeat.interval
> From: "3s"
> To: "5s"
> Error: For input string: "5s".
> {noformat}
> time unit support was added via HDFS-9847.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12085) Reconfigure namenode interval fails if the interval was set with time unit
Weiwei Yang created HDFS-12085:
--
Summary: Reconfigure namenode interval fails if the interval was
set with time unit
Key: HDFS-12085
URL: https://issues.apache.org/jira/browse/HDFS-12085
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs, tools
Reporter: Weiwei Yang
Assignee: Weiwei Yang
Priority: Critical
It fails when I set duration with time unit, e.g 5s, error
{noformat}
Reconfiguring status for node [localhost:8111]: started at Tue Jul 04 08:14:18
PDT 2017 and finished at Tue Jul 04 08:14:18 PDT 2017.
FAILED: Change property dfs.heartbeat.interval
From: "3s"
To: "5s"
Error: For input string: "5s".
{noformat}
time unit support was added via HDFS-9847.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11975) Provide a system-default EC policy
[
https://issues.apache.org/jira/browse/HDFS-11975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073840#comment-16073840
]
Hadoop QA commented on HDFS-11975:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2
extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
34s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
11s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 37s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 98m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11975 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875634/HDFS-11975-004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 7edfceff77bc 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20152/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
|
| findbugs |
[jira] [Commented] (HDFS-12084) Scheduled will not decrement when file is deleted before all IBR's received
[
https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073808#comment-16073808
]
Hadoop QA commented on HDFS-12084:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
45s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 36s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 4 new + 118 unchanged - 0 fixed = 122 total (was 118) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 29s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12084 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875631/HDFS-12084-001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 0d63622129d4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / b17e655 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20151/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20151/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20151/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20151/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20151/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Scheduled will not decrement
[jira] [Commented] (HDFS-11882) Client fails if acknowledged size is greater than bytes sent
[
https://issues.apache.org/jira/browse/HDFS-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073758#comment-16073758
]
Hadoop QA commented on HDFS-11882:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 38s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 55s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}120m 37s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure030 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-11882 |
| JIRA Patch URL |
[jira] [Commented] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073760#comment-16073760
]
Weiwei Yang commented on HDFS-12082:
Proposed a patch to fix this issue. Otherwise if user reconfigures namenode
interval, the value of property {{dfs.block.invalidate.limit}} will be always
overwritten.
The fix simply honors the configuration of {{dfs.block.invalidate.limit}} and
use it for block invalidate limit, and this will not change when heartbeat
interval changes. Reason is, following formula doesn't really work
{code}
// Default heartbeat is 3s, unless heartbeat is set to bigger than 50s,
// it is always 1000
(1) final int blockInvalidateLimit =
Math.max(20*(int)(heartbeatIntervalSeconds),
DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
// We will not reach the default value here, because we always load defaults
// from hdfs-default.xml. If the property is not found in hdfs-site.xml,
// it simply returns the default value from hdfs-default.xml, which is 1000.
// Even blockInvalidateLimit is something else, it doesn't count.
(2) this.blockInvalidateLimit = conf.getInt(
DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_KEY, blockInvalidateLimit);
{code}
so right now there are two cases
# {{dfs.block.invalidate.limit}} not set explicitly, blockInvalidateLimit=1000
# {{dfs.block.invalidate.limit}} is explicitly set,
blockInvalidateLimit=
in this case, why we still need (1) ? I think we can remove it.
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12082:
---
Priority: Critical (was: Major)
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Critical
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12082:
---
Priority: Major (was: Critical)
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12082:
---
Status: Patch Available (was: Open)
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
[
https://issues.apache.org/jira/browse/HDFS-12082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-12082:
---
Attachment: HDFS-12082.001.patch
> BlockInvalidateLimit value is incorrectly set after namenode heartbeat
> interval reconfigured
> -
>
> Key: HDFS-12082
> URL: https://issues.apache.org/jira/browse/HDFS-12082
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-12082.001.patch
>
>
> HDFS-1477 provides an option to reconfigured namenode heartbeat interval
> without restarting the namenode. When the heartbeat interval is reconfigured,
> {{blockInvalidateLimit}} gets recounted
> {code}
> this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
> DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
> {code}
> this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-11975) Provide a system-default EC policy
[
https://issues.apache.org/jira/browse/HDFS-11975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16071874#comment-16071874
]
luhuichun edited comment on HDFS-11975 at 7/4/17 1:27 PM:
--
[~eddyxu]，[~Sammi]，[~drankye] Update the patch, the failed unit test is solved.
For the issue of system-default EC policy, I add two const in
DFSConfigKeys.java. One for user setting, the other is hard code default value
"RS-6-3-64k". The NameNodeRpcServer do the judgement according to the
configuration and user CLI. Thanks for reviewing.
was (Author: luhuichun):
[~eddyxu][~Sammi][~drankye] Update the patch, the failed unit test is solved.
For the issue of system-default EC policy, I set a two configuration in
DFSConfigKeys.java. One for user setting, the other is hard code default value
"RS-6-3-64k". The NameNodeRpcServer do the judgement according to the
configuration and user CLI. Thanks for reviewing.
> Provide a system-default EC policy
> --
>
> Key: HDFS-11975
> URL: https://issues.apache.org/jira/browse/HDFS-11975
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: luhuichun
> Attachments: HDFS-11975-001.patch, HDFS-11975-002.patch,
> HDFS-11975-003.patch, HDFS-11975-004.patch
>
>
> From the usability point of view, it'd be nice to be able to specify a
> system-wide EC policy, i.e., in {{hdfs-site.xml}}. For most of users / admin
> / downstream projects, it is not necessary to know the tradeoffs of the EC
> policy, considering that it requires the knowledge of EC, the actual physical
> topology of the clusters, and many other factors (i.e., network, cluster size
> and etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11975) Provide a system-default EC policy
[
https://issues.apache.org/jira/browse/HDFS-11975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
luhuichun updated HDFS-11975:
-
Attachment: HDFS-11975-004.patch
> Provide a system-default EC policy
> --
>
> Key: HDFS-11975
> URL: https://issues.apache.org/jira/browse/HDFS-11975
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
>Affects Versions: 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: luhuichun
> Attachments: HDFS-11975-001.patch, HDFS-11975-002.patch,
> HDFS-11975-003.patch, HDFS-11975-004.patch
>
>
> From the usability point of view, it'd be nice to be able to specify a
> system-wide EC policy, i.e., in {{hdfs-site.xml}}. For most of users / admin
> / downstream projects, it is not necessary to know the tradeoffs of the EC
> policy, considering that it requires the knowledge of EC, the actual physical
> topology of the clusters, and many other factors (i.e., network, cluster size
> and etc).
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12084) Scheduled will not decrement when file is deleted before all IBR's received
[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-12084: Description: When small files creation && deletion happens so frequently and DN's did not report blocks to NN before deletion, then scheduled count will keep on increment and which will not deleted as blocks are deleted. *Note*: Every 20 mins,this can be rolled, but with in 20 mins, count can be more as so many operations. when batchIBR enabled with committed allowed=1 this will be observed more. was: When small files creation && deletion happens so frequently and DN's did not report blocks to NN before deletion, then scheduled size will keep on increment and which will not deleted as blocks are deleted. Note: Every 20 mins,this can be rolled, but with in 20 mins, size can be more as so many operations. when batchIBR enabled with committed allowed=1 this will be observed more. > Scheduled will not decrement when file is deleted before all IBR's received > --- > > Key: HDFS-12084 > URL: https://issues.apache.org/jira/browse/HDFS-12084 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HDFS-12084-001.patch > > > When small files creation && deletion happens so frequently and DN's did not > report blocks to NN before deletion, then scheduled count will keep on > increment and which will not deleted as blocks are deleted. > *Note*: Every 20 mins,this can be rolled, but with in 20 mins, count can be > more as so many operations. > when batchIBR enabled with committed allowed=1 this will be observed more. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12084) Scheduled will not decrement when file is deleted before all IBR's received
[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-12084: Status: Patch Available (was: Open) > Scheduled will not decrement when file is deleted before all IBR's received > --- > > Key: HDFS-12084 > URL: https://issues.apache.org/jira/browse/HDFS-12084 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HDFS-12084-001.patch > > > When small files creation && deletion happens so frequently and DN's did not > report blocks to NN before deletion, then scheduled size will keep on > increment and which will not deleted as blocks are deleted. > Note: Every 20 mins,this can be rolled, but with in 20 mins, size can be more > as so many operations. > when batchIBR enabled with committed allowed=1 this will be observed more. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12084) Scheduled will not decrement when file is deleted before all IBR's received
[ https://issues.apache.org/jira/browse/HDFS-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-12084: Attachment: HDFS-12084-001.patch Uploaded the patch,kindly review > Scheduled will not decrement when file is deleted before all IBR's received > --- > > Key: HDFS-12084 > URL: https://issues.apache.org/jira/browse/HDFS-12084 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HDFS-12084-001.patch > > > When small files creation && deletion happens so frequently and DN's did not > report blocks to NN before deletion, then scheduled size will keep on > increment and which will not deleted as blocks are deleted. > Note: Every 20 mins,this can be rolled, but with in 20 mins, size can be more > as so many operations. > when batchIBR enabled with committed allowed=1 this will be observed more. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12084) Scheduled will not decrement when file is deleted before all IBR's received
Brahma Reddy Battula created HDFS-12084: --- Summary: Scheduled will not decrement when file is deleted before all IBR's received Key: HDFS-12084 URL: https://issues.apache.org/jira/browse/HDFS-12084 Project: Hadoop HDFS Issue Type: Bug Components: namenode Reporter: Brahma Reddy Battula Assignee: Brahma Reddy Battula When small files creation && deletion happens so frequently and DN's did not report blocks to NN before deletion, then scheduled size will keep on increment and which will not deleted as blocks are deleted. Note: Every 20 mins,this can be rolled, but with in 20 mins, size can be more as so many operations. when batchIBR enabled with committed allowed=1 this will be observed more. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12083) Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys
[ https://issues.apache.org/jira/browse/HDFS-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12083: --- Priority: Critical (was: Major) > Ozone: KSM: previous key has to be excluded from result in listVolumes, > listBuckets and listKeys > > > Key: HDFS-12083 > URL: https://issues.apache.org/jira/browse/HDFS-12083 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar >Priority: Critical > > When previous key is set as part of list calls [listVolume, listBuckets & > listKeys], the result includes previous key, there is no need to have this in > the result. > Since previous key is present as part of result, we will never receive an > empty list in the subsequent list calls, this makes it difficult to have a > exit criteria where we want to get all the values using multiple list calls > (with previous-key set). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12083) Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys
[
https://issues.apache.org/jira/browse/HDFS-12083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073643#comment-16073643
]
Weiwei Yang commented on HDFS-12083:
Hi [~nandakumar131]
Thanks for filing this. I agree. From pagination point of view, it is easier to
use prev-key in the listing result, for example we have keys
{noformat}a, b, c, d, e{noformat}
so to get multi-page result, we can use following calls
{code}
listKeys -> prev-key=null, max-keys=2 -> a, b
listKeys -> prev-key=b, max-keys=2 -> c, d
listKeys -> prev-key=d, max-keys=2 -> e
{code}
when you work on this, please also keep our java doc (for listing methods) and
documentation ({{OzoneRest.md}}) up-to-date.
Thanks a lot!
> Ozone: KSM: previous key has to be excluded from result in listVolumes,
> listBuckets and listKeys
>
>
> Key: HDFS-12083
> URL: https://issues.apache.org/jira/browse/HDFS-12083
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nandakumar
>Assignee: Nandakumar
>
> When previous key is set as part of list calls [listVolume, listBuckets &
> listKeys], the result includes previous key, there is no need to have this in
> the result.
> Since previous key is present as part of result, we will never receive an
> empty list in the subsequent list calls, this makes it difficult to have a
> exit criteria where we want to get all the values using multiple list calls
> (with previous-key set).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11882) Client fails if acknowledged size is greater than bytes sent
[
https://issues.apache.org/jira/browse/HDFS-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073598#comment-16073598
]
Akira Ajisaka commented on HDFS-11882:
--
Now I added the test case to TestDFSStripedOutputStreamWithFailure, however,
it's not good because the test case will be run in the existing subclasses. I'm
not sure where to add. Now I'm thinking it's good to add a subclass to ignore
the test case and modify the other existing subclasses to extend the subclass.
That way we can easily add test cases to TestDFSStripledOuputStream without any
effect on the existing subclasses.
> Client fails if acknowledged size is greater than bytes sent
>
>
> Key: HDFS-11882
> URL: https://issues.apache.org/jira/browse/HDFS-11882
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding, test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Critical
> Labels: hdfs-ec-3.0-nice-to-have
> Attachments: HDFS-11882.01.patch, HDFS-11882.regressiontest.patch
>
>
> Some tests of erasure coding fails by the following exception. The following
> test was removed by HDFS-11823, however, this type of error can happen in
> real cluster.
> {noformat}
> Running
> org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 10, Time elapsed: 89.086 sec
> <<< FAILURE! - in
> org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
> testMultipleDatanodeFailure56(org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure)
> Time elapsed: 38.831 sec <<< ERROR!
> java.lang.IllegalStateException: null
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.updatePipeline(DFSStripedOutputStream.java:780)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:664)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.closeImpl(DFSStripedOutputStream.java:1034)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:842)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:472)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTestWithMultipleFailure(TestDFSStripedOutputStreamWithFailure.java:381)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.testMultipleDatanodeFailure56(TestDFSStripedOutputStreamWithFailure.java:245)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11882) Client fails if acknowledged size is greater than bytes sent
[
https://issues.apache.org/jira/browse/HDFS-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-11882:
-
Attachment: HDFS-11882.regressiontest.patch
Attaching a patch to add a regression test for this issue.
> Client fails if acknowledged size is greater than bytes sent
>
>
> Key: HDFS-11882
> URL: https://issues.apache.org/jira/browse/HDFS-11882
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding, test
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Critical
> Labels: hdfs-ec-3.0-nice-to-have
> Attachments: HDFS-11882.01.patch, HDFS-11882.regressiontest.patch
>
>
> Some tests of erasure coding fails by the following exception. The following
> test was removed by HDFS-11823, however, this type of error can happen in
> real cluster.
> {noformat}
> Running
> org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
> Tests run: 14, Failures: 0, Errors: 1, Skipped: 10, Time elapsed: 89.086 sec
> <<< FAILURE! - in
> org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
> testMultipleDatanodeFailure56(org.apache.hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure)
> Time elapsed: 38.831 sec <<< ERROR!
> java.lang.IllegalStateException: null
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.updatePipeline(DFSStripedOutputStream.java:780)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.checkStreamerFailures(DFSStripedOutputStream.java:664)
> at
> org.apache.hadoop.hdfs.DFSStripedOutputStream.closeImpl(DFSStripedOutputStream.java:1034)
> at
> org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:842)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:101)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTest(TestDFSStripedOutputStreamWithFailure.java:472)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.runTestWithMultipleFailure(TestDFSStripedOutputStreamWithFailure.java:381)
> at
> org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure.testMultipleDatanodeFailure56(TestDFSStripedOutputStreamWithFailure.java:245)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12083) Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys
Nandakumar created HDFS-12083: - Summary: Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys Key: HDFS-12083 URL: https://issues.apache.org/jira/browse/HDFS-12083 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Reporter: Nandakumar Assignee: Nandakumar When previous key is set as part of list calls [listVolume, listBuckets & listKeys], the result includes previous key, there is no need to have this in the result. Since previous key is present as part of result, we will never receive an empty list in the subsequent list calls, this makes it difficult to have a exit criteria where we want to get all the values using multiple list calls (with previous-key set). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-12082) BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured
Weiwei Yang created HDFS-12082:
--
Summary: BlockInvalidateLimit value is incorrectly set after
namenode heartbeat interval reconfigured
Key: HDFS-12082
URL: https://issues.apache.org/jira/browse/HDFS-12082
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs, namenode
Reporter: Weiwei Yang
Assignee: Weiwei Yang
HDFS-1477 provides an option to reconfigured namenode heartbeat interval
without restarting the namenode. When the heartbeat interval is reconfigured,
{{blockInvalidateLimit}} gets recounted
{code}
this.blockInvalidateLimit = Math.max(20 * (int) (intervalSeconds),
DFSConfigKeys.DFS_BLOCK_INVALIDATE_LIMIT_DEFAULT);
{code}
this doesn't honor the existing value set by {{dfs.block.invalidate.limit}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12067) Add command help description about 'hdfs dfsadmin -help getVolumeReport' command.
[
https://issues.apache.org/jira/browse/HDFS-12067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073392#comment-16073392
]
Hadoop QA commented on HDFS-12067:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
39s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 37s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12067 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12875582/HDFS_12067.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 36a4901c3928 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 186650d |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/20149/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20149/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20149/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20149/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add command help description about 'hdfs dfsadmin -help getVolumeReport'
> command.
> -
>
>
[jira] [Updated] (HDFS-12067) Add command help description about 'hdfs dfsadmin -help getVolumeReport' command.
[ https://issues.apache.org/jira/browse/HDFS-12067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] steven-wugang updated HDFS-12067: - Status: Patch Available (was: Open) > Add command help description about 'hdfs dfsadmin -help getVolumeReport' > command. > - > > Key: HDFS-12067 > URL: https://issues.apache.org/jira/browse/HDFS-12067 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: steven-wugang >Assignee: steven-wugang > Attachments: HDFS_12067.001.patch, HDFS_12067.002.patch, > HDFS_12067.003.patch, HDFS_12067.004.patch, HDFS-12067.patch > > > When I use the command,I see the command help description,but the help > description doesn't make it clear,especially the argument 'port',It's easy to > mistake for port (default 9866) in 'dfs.datanode.address'.Therefore, in order > to use this command better,I add some descriptions about the arguments. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12067) Add command help description about 'hdfs dfsadmin -help getVolumeReport' command.
[ https://issues.apache.org/jira/browse/HDFS-12067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] steven-wugang updated HDFS-12067: - Status: Open (was: Patch Available) > Add command help description about 'hdfs dfsadmin -help getVolumeReport' > command. > - > > Key: HDFS-12067 > URL: https://issues.apache.org/jira/browse/HDFS-12067 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: steven-wugang >Assignee: steven-wugang > Attachments: HDFS_12067.001.patch, HDFS_12067.002.patch, > HDFS_12067.003.patch, HDFS_12067.004.patch, HDFS-12067.patch > > > When I use the command,I see the command help description,but the help > description doesn't make it clear,especially the argument 'port',It's easy to > mistake for port (default 9866) in 'dfs.datanode.address'.Therefore, in order > to use this command better,I add some descriptions about the arguments. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12067) Add command help description about 'hdfs dfsadmin -help getVolumeReport' command.
[ https://issues.apache.org/jira/browse/HDFS-12067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] steven-wugang updated HDFS-12067: - Attachment: HDFS_12067.004.patch > Add command help description about 'hdfs dfsadmin -help getVolumeReport' > command. > - > > Key: HDFS-12067 > URL: https://issues.apache.org/jira/browse/HDFS-12067 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: steven-wugang >Assignee: steven-wugang > Attachments: HDFS_12067.001.patch, HDFS_12067.002.patch, > HDFS_12067.003.patch, HDFS_12067.004.patch, HDFS-12067.patch > > > When I use the command,I see the command help description,but the help > description doesn't make it clear,especially the argument 'port',It's easy to > mistake for port (default 9866) in 'dfs.datanode.address'.Therefore, in order > to use this command better,I add some descriptions about the arguments. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12078) Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml
[
https://issues.apache.org/jira/browse/HDFS-12078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073168#comment-16073168
]
Hudson commented on HDFS-12078:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11964 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11964/])
HDFS-12078. Add time unit to the description of property (aajisaka: rev
186650d21d482e2f3bc4523ae989ebe76081b0e3)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
> Add time unit to the description of property
> dfs.namenode.stale.datanode.interval in hdfs-default.xml
> -
>
> Key: HDFS-12078
> URL: https://issues.apache.org/jira/browse/HDFS-12078
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, hdfs
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Minor
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: HDFS-12078.001.patch
>
>
> The description of property {{dfs.namenode.stale.datanode.interval}} in
> hdfs-default.xml doesn't mention about the time unit, we should add that to
> avoid confusing users.
> I have reviewed all properties in hdfs-default.xml, this is the only one
> property causes such confusion, user should be able to easily figure out the
> appropriate time unit for properties by either
> * Specified by property name, e.g
> dfs.namenode.full.block.report.lease.length.ms
> * Specified by property value with time unit suffix, e.g
> dfs.blockreport.initialDelay=0s
> * Explained by description of the property, e.g
> dfs.namenode.safemode.extension=3000, description: Determines extension of
> safe mode in milliseconds ...
> change to the property name and value will be an incompatible change, to
> minimize the impact, propose to simply add the time unit in the description
> (3rd approach). And this should be only one property needs the fix in
> hdfs-default.xml.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12069) Ozone: Create a general abstraction for metadata store
[ https://issues.apache.org/jira/browse/HDFS-12069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073166#comment-16073166 ] Mukul Kumar Singh commented on HDFS-12069: -- Thanks for working on this [~cheersyang], the patch looks good I have some minor comments 1) Can the config changes in OzoneConfigKeys.java be move to SCMConfig keys or another common location ? This is needed so that the abstracted MetadataStore can also be used by CBlock and other related subsystems. 2) LevelDBStore.java:81, Can we log the DB Path and the options if open fails ? 3) LevelDBStore.java:261, I am not really sure about this, but I feel that if there is only one operation inside the batch then the operation can be called directly without creating a batch. This will help reduce the batching overhead. > Ozone: Create a general abstraction for metadata store > -- > > Key: HDFS-12069 > URL: https://issues.apache.org/jira/browse/HDFS-12069 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-12069-HDFS-7240.001.patch, > HDFS-12069-HDFS-7240.002.patch, HDFS-12069-HDFS-7240.003.patch > > > Create a general abstraction for metadata store so that we can plug other key > value store to host ozone metadata. Currently only levelDB is implemented, we > want to support RocksDB as it provides more production ready features. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
