[jira] [Updated] (HADOOP-14689) PreCommit-HADOOP-Build on branch-2.7 is failing.
[ https://issues.apache.org/jira/browse/HADOOP-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HADOOP-14689: - Summary: PreCommit-HADOOP-Build on branch-2.7 is failing. (was: Build on branch-2.7 is failing.) > PreCommit-HADOOP-Build on branch-2.7 is failing. > > > Key: HADOOP-14689 > URL: https://issues.apache.org/jira/browse/HADOOP-14689 > Project: Hadoop Common > Issue Type: Task >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Labels: build > Attachments: HADOOP-14689-branch-2.7.001.patch > > > Will add dummy patches to test Jenkins PreCommit-HADOOP-Build failures on > branch-2.7. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14689) Build on branch-2.7 is failing.
[ https://issues.apache.org/jira/browse/HADOOP-14689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HADOOP-14689: - Summary: Build on branch-2.7 is failing. (was: Dummy jira for testing barnch-2.7 build.) > Build on branch-2.7 is failing. > --- > > Key: HADOOP-14689 > URL: https://issues.apache.org/jira/browse/HADOOP-14689 > Project: Hadoop Common > Issue Type: Task >Reporter: Konstantin Shvachko >Assignee: Konstantin Shvachko > Labels: build > Attachments: HADOOP-14689-branch-2.7.001.patch > > > Will add dummy patches to test Jenkins PreCommit-HADOOP-Build failures on > branch-2.7. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108403#comment-16108403
]
Konstantin Shvachko commented on HADOOP-14676:
--
+1 Looks good.
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[
https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108377#comment-16108377
]
Hadoop QA commented on HADOOP-14673:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
10s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
30s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14673 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879767/HADOOP-14673.01.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux 9a296a684321 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a4aa1cb |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12909/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12909/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Remove leftover hadoop_xml_escape from functions
>
>
> Key: HADOOP-14673
> URL: https://issues.apache.org/jira/browse/HADOOP-14673
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 3.0.0-beta1
>Reporter: Allen Wittenauer
>Assignee: Ajay Yadav
> Attachments: HADOOP-14673.01.patch
>
>
> This function is no longer needed, so let's purge it.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[ https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HADOOP-14673: Status: Patch Available (was: Open) > Remove leftover hadoop_xml_escape from functions > > > Key: HADOOP-14673 > URL: https://issues.apache.org/jira/browse/HADOOP-14673 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0-beta1 >Reporter: Allen Wittenauer >Assignee: Ajay Yadav > Attachments: HADOOP-14673.01.patch > > > This function is no longer needed, so let's purge it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14705) Add batched reencryptEncryptedKey interface to KMS
[
https://issues.apache.org/jira/browse/HADOOP-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HADOOP-14705:
---
Attachment: HADOOP-14705.01.patch
Patch 1 to express the idea and for high-level review.
Will add more test cases, and look to see if some of the util methods can be
de-dup'ed.
> Add batched reencryptEncryptedKey interface to KMS
> --
>
> Key: HADOOP-14705
> URL: https://issues.apache.org/jira/browse/HADOOP-14705
> Project: Hadoop Common
> Issue Type: Improvement
> Components: kms
>Reporter: Xiao Chen
>Assignee: Xiao Chen
> Attachments: HADOOP-14705.01.patch
>
>
> HADOOP-13827 already enabled the KMS to re-encrypt a {{EncryptedKeyVersion}}.
> But as the performance results of HDFS-10899 turns out, communication
> overhead with the KMS occupies the majority of the time. So we need a batched
> interface to re-encrypt multiple EDEKs in 1 call.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14705) Add batched reencryptEncryptedKey interface to KMS
Xiao Chen created HADOOP-14705:
--
Summary: Add batched reencryptEncryptedKey interface to KMS
Key: HADOOP-14705
URL: https://issues.apache.org/jira/browse/HADOOP-14705
Project: Hadoop Common
Issue Type: Improvement
Components: kms
Reporter: Xiao Chen
Assignee: Xiao Chen
HADOOP-13827 already enabled the KMS to re-encrypt a {{EncryptedKeyVersion}}.
But as the performance results of HDFS-10899 turns out, communication overhead
with the KMS occupies the majority of the time. So we need a batched interface
to re-encrypt multiple EDEKs in 1 call.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108343#comment-16108343
]
Hudson commented on HADOOP-14397:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12087 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12087/])
HADOOP-14397. Pull up the builder pattern to FileSystem and add (lei: rev
9586b0e24fce29c278134658e68b8c47cd9d8c51)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractAppendTest.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractCreateTest.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataOutputStreamBuilder.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/ContractTestUtils.java
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch, HADOOP-14397.004.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[ https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108333#comment-16108333 ] Ajay Yadav edited comment on HADOOP-14673 at 8/1/17 3:45 AM: - [~aw], Removed functions from hadoop-functions.sh and related test cases . Could you please review the patch? was (Author: ajayydv): [~aw], Removed functions from hadoop-functions.sh and related test cases . Could you please the patch? > Remove leftover hadoop_xml_escape from functions > > > Key: HADOOP-14673 > URL: https://issues.apache.org/jira/browse/HADOOP-14673 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0-beta1 >Reporter: Allen Wittenauer >Assignee: Ajay Yadav > Attachments: HADOOP-14673.01.patch > > > This function is no longer needed, so let's purge it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[ https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav updated HADOOP-14673: Attachment: HADOOP-14673.01.patch > Remove leftover hadoop_xml_escape from functions > > > Key: HADOOP-14673 > URL: https://issues.apache.org/jira/browse/HADOOP-14673 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0-beta1 >Reporter: Allen Wittenauer > Attachments: HADOOP-14673.01.patch > > > This function is no longer needed, so let's purge it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[ https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Yadav reassigned HADOOP-14673: --- Assignee: Ajay Yadav > Remove leftover hadoop_xml_escape from functions > > > Key: HADOOP-14673 > URL: https://issues.apache.org/jira/browse/HADOOP-14673 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0-beta1 >Reporter: Allen Wittenauer >Assignee: Ajay Yadav > Attachments: HADOOP-14673.01.patch > > > This function is no longer needed, so let's purge it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14673) Remove leftover hadoop_xml_escape from functions
[ https://issues.apache.org/jira/browse/HADOOP-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108333#comment-16108333 ] Ajay Yadav commented on HADOOP-14673: - [~aw], Removed functions from hadoop-functions.sh and related test cases . Could you please the patch? > Remove leftover hadoop_xml_escape from functions > > > Key: HADOOP-14673 > URL: https://issues.apache.org/jira/browse/HADOOP-14673 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts >Affects Versions: 3.0.0-beta1 >Reporter: Allen Wittenauer > > This function is no longer needed, so let's purge it. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14343) Wrong pid file name in error message when starting secure daemon
[
https://issues.apache.org/jira/browse/HADOOP-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108327#comment-16108327
]
Hudson commented on HADOOP-14343:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12086 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12086/])
HADOOP-14343. Wrong pid file name in error message when starting secure (aw:
rev abbf4129a24c99fbce6d70b191ec19cf0d17e9be)
* (edit) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
> Wrong pid file name in error message when starting secure daemon
>
>
> Key: HADOOP-14343
> URL: https://issues.apache.org/jira/browse/HADOOP-14343
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14343.01.patch, HADOOP-14343.02.patch
>
>
> {code}# this is for the daemon pid creation
> #shellcheck disable=SC2086
> echo $! > "${jsvcpidfile}" 2>/dev/null
> if [[ $? -gt 0 ]]; then
> hadoop_error "ERROR: Cannot write ${daemonname} pid ${daemonpidfile}."
> fi{code}
> It will log datanode's pid file instead of JSVC's pid file.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HADOOP-14397:
---
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
2.9.0
Status: Resolved (was: Patch Available)
Thanks [~manojg] for reviews.
Committed to trunk and branch-2.
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch, HADOOP-14397.004.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14667) Flexible Visual Studio support
[ https://issues.apache.org/jira/browse/HADOOP-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108315#comment-16108315 ] Allen Wittenauer commented on HADOOP-14667: --- OK, I'll try to get to it this week. The more I play around with the hadoop-trunk-win build, the more I'm convinced that it's the only way to make it work semi-reliably in "build the world" mode, never mind actual patch testing. > Flexible Visual Studio support > -- > > Key: HADOOP-14667 > URL: https://issues.apache.org/jira/browse/HADOOP-14667 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-beta1 > Environment: Windows >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-14667.00.patch, HADOOP-14667.01.patch, > HADOOP-14667.02.patch, HADOOP-14667.03.patch, HADOOP-14667.04.patch > > > Is it time to upgrade the Windows native project files to use something more > modern than Visual Studio 2010? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14343) Wrong pid file name in error message when starting secure daemon
[
https://issues.apache.org/jira/browse/HADOOP-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-14343:
--
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
Status: Resolved (was: Patch Available)
+1 thanks committed to trunk
> Wrong pid file name in error message when starting secure daemon
>
>
> Key: HADOOP-14343
> URL: https://issues.apache.org/jira/browse/HADOOP-14343
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14343.01.patch, HADOOP-14343.02.patch
>
>
> {code}# this is for the daemon pid creation
> #shellcheck disable=SC2086
> echo $! > "${jsvcpidfile}" 2>/dev/null
> if [[ $? -gt 0 ]]; then
> hadoop_error "ERROR: Cannot write ${daemonname} pid ${daemonpidfile}."
> fi{code}
> It will log datanode's pid file instead of JSVC's pid file.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[
https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108258#comment-16108258
]
Andrew Wang commented on HADOOP-14701:
--
Thanks for filing this JIRA and providing a patch [~asherman]! Your detailed
explanation made it quite clear, the scoping of the field and parameter do not
match.
One question, does it log the WARN when a property is declared multiple times
in the same file? Wasn't clear when {{this.properties}} gets updated. A test
for this would be nice too.
> Configuration can log misleading warnings about an attempt to override final
> parameter
> --
>
> Key: HADOOP-14701
> URL: https://issues.apache.org/jira/browse/HADOOP-14701
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Andrew Sherman
> Attachments: HADOOP-14701.1.patch, HADOOP-14701.2.patch,
> HADOOP-14701.3.patch
>
>
> The class Configuration (in hadoop-common-project) is used to hold
> configuration properties.
> The method loadResources() is used to load properties from a list of default
> resources into a parameter called “properties” (which happens to always be
> the field called “properties”).
> This is done by calling the method loadResource() which is used to load
> properties from a single resource into a parameter called “properties”
> (which happens to always be the field called “properties”).
> This is done by calling loadProperty() which is used to load a single
> property into a parameter called “properties” (which is sometimes, but not
> always, the field called “properties”).
> In loadProperty() there is some logic for dealing with properties that are
> marked as final.
> (1) a final property is not set into the “properties” parameter.
> (2) a warning is printed for a final property if the value of the property
> with the same name in the “properties” parameter is not equal to the new
> value, and the property name is in the finalParameters field.
> When loadResource() is reading properties from a stream it creates a
> temporary Properties object that it populates by calling loadProperty(). This
> is the case where loadProperty() has a parameter called “properties” which
> is not the same as the field called “properties”).
> When loadProperty() is called on a final property then because of (1) it is
> not set into the “properties” parameter. But sometimes the property name is
> already in the finalParameters field and now (2) the warning is triggered
> because the property is not present in the “properties” parameter (although
> it IS present in the “properties” field). The warning is misleading and
> confusing.
> h3. Possible Fix
> One way of looking at the problem is that loadProperty is using the
> finalParameters field in combination with the “properties” parameter. Change
> the code to only generate a warning for a final property if the value of the
> property with the same name in the “properties” field is not equal to the new
> value, and the property name is in the finalParameters field.
> I’ve made a patch containing this tiny change.
> Note that I have done very minimal testing of this fix
> h3. Example warning:
> 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration:
> [HiveServer2-Background-Pool: Thread-77]:
> hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14519) Client$Connection#waitForWork may suffer from spurious wakeups
[
https://issues.apache.org/jira/browse/HADOOP-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108250#comment-16108250
]
John Zhuge commented on HADOOP-14519:
-
Thanks [~vagarychen] and [~djp] for the comments. I will get back to you.
> Client$Connection#waitForWork may suffer from spurious wakeups
> --
>
> Key: HADOOP-14519
> URL: https://issues.apache.org/jira/browse/HADOOP-14519
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Critical
> Attachments: HADOOP-14519.001.patch
>
>
> {{Client$Connection#waitForWork}} may suffer spurious wakeup because the
> {{wait}} is not surrounded by a loop. See
> [https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#wait()].
> {code:title=Client$Connection#waitForWork}
> if (calls.isEmpty() && !shouldCloseConnection.get() && running.get()) {
> long timeout = maxIdleTime-
> (Time.now()-lastActivity.get());
> if (timeout>0) {
> try {
> wait(timeout); << spurious wakeup
> } catch (InterruptedException e) {}
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8
[ https://issues.apache.org/jira/browse/HADOOP-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108249#comment-16108249 ] Junping Du commented on HADOOP-14683: - Thanks for confirmation, Andrew. +1 on 02 patch. Will commit it after 24 hours if not further comments on latest patch. > FileStatus.compareTo binary compat issue between 2.7 and 2.8 > > > Key: HADOOP-14683 > URL: https://issues.apache.org/jira/browse/HADOOP-14683 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.8.1 >Reporter: Sergey Shelukhin >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: HADOOP-14683-branch-2-01.patch, > HADOOP-14683-branch-2-02.patch > > > See HIVE-17133. Looks like the signature change is causing issues; according > to [~jnp] this is a public API. > Is it possible to add the old overload back (keeping the new one presumably) > in a point release on 2.8? That way we can avoid creating yet another shim > for this in Hive. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14519) Client$Connection#waitForWork may suffer from spurious wakeups
[
https://issues.apache.org/jira/browse/HADOOP-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108244#comment-16108244
]
Junping Du commented on HADOOP-14519:
-
I think adding while loop here intention is to get rid of spurious
notification. So get notified and check condition A should be good. One thing
we could be missing here is we could miss some notification when hitting
condition A. First two conditions should be fine as we send notifications when
we receive new calls or Close Connection. However, when we do stop() - where we
set running to false, it sounds like we are missing to send notification. May
be safe to add notification?
Other looks good to me.
> Client$Connection#waitForWork may suffer from spurious wakeups
> --
>
> Key: HADOOP-14519
> URL: https://issues.apache.org/jira/browse/HADOOP-14519
> Project: Hadoop Common
> Issue Type: Bug
> Components: ipc
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Critical
> Attachments: HADOOP-14519.001.patch
>
>
> {{Client$Connection#waitForWork}} may suffer spurious wakeup because the
> {{wait}} is not surrounded by a loop. See
> [https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#wait()].
> {code:title=Client$Connection#waitForWork}
> if (calls.isEmpty() && !shouldCloseConnection.get() && running.get()) {
> long timeout = maxIdleTime-
> (Time.now()-lastActivity.get());
> if (timeout>0) {
> try {
> wait(timeout); << spurious wakeup
> } catch (InterruptedException e) {}
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108235#comment-16108235
]
Manoj Govindassamy commented on HADOOP-14397:
-
Looks good to me, [~eddyxu]. +1.
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch, HADOOP-14397.004.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14389) Exception handling is incorrect in KerberosName.java
[
https://issues.apache.org/jira/browse/HADOOP-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108234#comment-16108234
]
Hadoop QA commented on HADOOP-14389:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hadoop-auth in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
39s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 39s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} hadoop-common-project/hadoop-auth: The patch
generated 0 new + 0 unchanged - 8 fixed = 0 total (was 8) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
22s{color} | {color:red} hadoop-auth in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} hadoop-auth in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 18s{color}
| {color:red} hadoop-auth in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14389 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12866648/HADOOP-14389.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 9319fb190eca 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ea56812 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-auth.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/patch-compile-root.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/patch-compile-root.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-auth.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/whitespace-eol.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12908/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-auth.txt
|
| unit |
[jira] [Commented] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[
https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108233#comment-16108233
]
Hadoop QA commented on HADOOP-14701:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 1s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14701 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879743/HADOOP-14701.3.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 18ce03249b7d 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ea56812 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12907/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12907/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12907/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Configuration can log misleading warnings about an attempt to override final
> parameter
> --
>
> Key: HADOOP-14701
> URL: https://issues.apache.org/jira/browse/HADOOP-14701
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Andrew Sherman
> Attachments:
[jira] [Commented] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8
[ https://issues.apache.org/jira/browse/HADOOP-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108226#comment-16108226 ] Andrew Wang commented on HADOOP-14683: -- Thanks for the ping, looks like a good one to target for 3.0.0-beta1. I'll note that binary compatibility between 2.x and 3.0.0 is a non-goal, but I'm all for making it the transition as smooth as possible. > FileStatus.compareTo binary compat issue between 2.7 and 2.8 > > > Key: HADOOP-14683 > URL: https://issues.apache.org/jira/browse/HADOOP-14683 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.8.1 >Reporter: Sergey Shelukhin >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: HADOOP-14683-branch-2-01.patch, > HADOOP-14683-branch-2-02.patch > > > See HIVE-17133. Looks like the signature change is causing issues; according > to [~jnp] this is a public API. > Is it possible to add the old overload back (keeping the new one presumably) > in a point release on 2.8? That way we can avoid creating yet another shim > for this in Hive. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8
[ https://issues.apache.org/jira/browse/HADOOP-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-14683: - Target Version/s: 2.9.0, 3.0.0-beta1, 2.8.2 (was: 2.9.0, 2.8.2) > FileStatus.compareTo binary compat issue between 2.7 and 2.8 > > > Key: HADOOP-14683 > URL: https://issues.apache.org/jira/browse/HADOOP-14683 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.8.1 >Reporter: Sergey Shelukhin >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: HADOOP-14683-branch-2-01.patch, > HADOOP-14683-branch-2-02.patch > > > See HIVE-17133. Looks like the signature change is causing issues; according > to [~jnp] this is a public API. > Is it possible to add the old overload back (keeping the new one presumably) > in a point release on 2.8? That way we can avoid creating yet another shim > for this in Hive. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14683) FileStatus.compareTo binary compat issue between 2.7 and 2.8
[ https://issues.apache.org/jira/browse/HADOOP-14683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108221#comment-16108221 ] Junping Du commented on HADOOP-14683: - bq. Generally yes, but if we use instanceof, we need to throw some Exception explicitly, which is the same as casting without type check. I'll update javadoc to add @throws ClassCastException. I think add javadoc should be fine given we cast blindly in other places, like: Path, SequenceFile, etc. 02 patch LGTM. I think we should fix in trunk and 3.0-beta also. CC [~andrew.wang]. > FileStatus.compareTo binary compat issue between 2.7 and 2.8 > > > Key: HADOOP-14683 > URL: https://issues.apache.org/jira/browse/HADOOP-14683 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.8.0, 2.8.1 >Reporter: Sergey Shelukhin >Assignee: Akira Ajisaka >Priority: Blocker > Attachments: HADOOP-14683-branch-2-01.patch, > HADOOP-14683-branch-2-02.patch > > > See HIVE-17133. Looks like the signature change is causing issues; according > to [~jnp] this is a public API. > Is it possible to add the old overload back (keeping the new one presumably) > in a point release on 2.8? That way we can avoid creating yet another shim > for this in Hive. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14389) Exception handling is incorrect in KerberosName.java
[
https://issues.apache.org/jira/browse/HADOOP-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108197#comment-16108197
]
John Zhuge commented on HADOOP-14389:
-
[~boky01] Thanks for enhancing the supportability of Kerberos name rules.
Tried to compile with the patch but go this error:
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure
[ERROR]
/Users/jzhuge/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/KerberosName.java:[336,13]
cannot find symbol
[ERROR] symbol: method splitToList(java.lang.String)
[ERROR] location: class com.google.common.base.Splitter
{noformat}
Also line #256 exceeded 80 characters.
> Exception handling is incorrect in KerberosName.java
>
>
> Key: HADOOP-14389
> URL: https://issues.apache.org/jira/browse/HADOOP-14389
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Labels: supportability
> Attachments: HADOOP-14389.01.patch
>
>
> I found multiple inconsistency:
> Rule: {{RULE:\[2:$1/$2\@$3\](.\*)s/.\*/hdfs/}}
> Principal: {{nn/host.dom...@realm.tld}}
> Expected exception: {{BadStringFormat: ...3 is out of range...}}
> Actual exception: {{ArrayIndexOutOfBoundsException: 3}}
>
> Rule: {{RULE:\[:$1/$2\@$0](.\*)s/.\*/hdfs/}} (Missing num of components)
> Expected: {{IllegalArgumentException}}
> Actual: {{java.lang.NumberFormatException: For input string: ""}}
>
> Rule: {{RULE:\[2:$-1/$2\@$3\](.\*)s/.\*/hdfs/}}
> Expected {{BadStringFormat: -1 is outside of valid range...}}
> Actual: {{java.lang.NumberFormatException: For input string: ""}}
>
> Rule: {{RULE:\[2:$one/$2\@$3\](.\*)s/.\*/hdfs/}}
> Expected {{java.lang.NumberFormatException: For input string: "one"}}
> Acutal {{java.lang.NumberFormatException: For input string: ""}}
>
> In addtion:
> {code}[^\\]]{code}
> does not really make sense in {{ruleParser}}. Most probably it was needed
> because we parse the whole rule string and remove the parsed rule from
> beginning of the string: {{KerberosName#parseRules}}. This made the regex
> engine parse wrong without it.
> In addition:
> In tests some corner cases are not covered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14389) Exception handling is incorrect in KerberosName.java
[
https://issues.apache.org/jira/browse/HADOOP-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HADOOP-14389:
Labels: supportability (was: )
> Exception handling is incorrect in KerberosName.java
>
>
> Key: HADOOP-14389
> URL: https://issues.apache.org/jira/browse/HADOOP-14389
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Labels: supportability
> Attachments: HADOOP-14389.01.patch
>
>
> I found multiple inconsistency:
> Rule: {{RULE:\[2:$1/$2\@$3\](.\*)s/.\*/hdfs/}}
> Principal: {{nn/host.dom...@realm.tld}}
> Expected exception: {{BadStringFormat: ...3 is out of range...}}
> Actual exception: {{ArrayIndexOutOfBoundsException: 3}}
>
> Rule: {{RULE:\[:$1/$2\@$0](.\*)s/.\*/hdfs/}} (Missing num of components)
> Expected: {{IllegalArgumentException}}
> Actual: {{java.lang.NumberFormatException: For input string: ""}}
>
> Rule: {{RULE:\[2:$-1/$2\@$3\](.\*)s/.\*/hdfs/}}
> Expected {{BadStringFormat: -1 is outside of valid range...}}
> Actual: {{java.lang.NumberFormatException: For input string: ""}}
>
> Rule: {{RULE:\[2:$one/$2\@$3\](.\*)s/.\*/hdfs/}}
> Expected {{java.lang.NumberFormatException: For input string: "one"}}
> Acutal {{java.lang.NumberFormatException: For input string: ""}}
>
> In addtion:
> {code}[^\\]]{code}
> does not really make sense in {{ruleParser}}. Most probably it was needed
> because we parse the whole rule string and remove the parsed rule from
> beginning of the string: {{KerberosName#parseRules}}. This made the regex
> engine parse wrong without it.
> In addition:
> In tests some corner cases are not covered.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[ https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HADOOP-14701: Attachment: HADOOP-14701.3.patch > Configuration can log misleading warnings about an attempt to override final > parameter > -- > > Key: HADOOP-14701 > URL: https://issues.apache.org/jira/browse/HADOOP-14701 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Reporter: Andrew Sherman > Attachments: HADOOP-14701.1.patch, HADOOP-14701.2.patch, > HADOOP-14701.3.patch > > > The class Configuration (in hadoop-common-project) is used to hold > configuration properties. > The method loadResources() is used to load properties from a list of default > resources into a parameter called “properties” (which happens to always be > the field called “properties”). > This is done by calling the method loadResource() which is used to load > properties from a single resource into a parameter called “properties” > (which happens to always be the field called “properties”). > This is done by calling loadProperty() which is used to load a single > property into a parameter called “properties” (which is sometimes, but not > always, the field called “properties”). > In loadProperty() there is some logic for dealing with properties that are > marked as final. > (1) a final property is not set into the “properties” parameter. > (2) a warning is printed for a final property if the value of the property > with the same name in the “properties” parameter is not equal to the new > value, and the property name is in the finalParameters field. > When loadResource() is reading properties from a stream it creates a > temporary Properties object that it populates by calling loadProperty(). This > is the case where loadProperty() has a parameter called “properties” which > is not the same as the field called “properties”). > When loadProperty() is called on a final property then because of (1) it is > not set into the “properties” parameter. But sometimes the property name is > already in the finalParameters field and now (2) the warning is triggered > because the property is not present in the “properties” parameter (although > it IS present in the “properties” field). The warning is misleading and > confusing. > h3. Possible Fix > One way of looking at the problem is that loadProperty is using the > finalParameters field in combination with the “properties” parameter. Change > the code to only generate a warning for a final property if the value of the > property with the same name in the “properties” field is not equal to the new > value, and the property name is in the finalParameters field. > I’ve made a patch containing this tiny change. > Note that I have done very minimal testing of this fix > h3. Example warning: > 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration: > [HiveServer2-Background-Pool: Thread-77]: > hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[
https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108157#comment-16108157
]
Hadoop QA commented on HADOOP-14701:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
9s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 38s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 9 new + 262 unchanged - 0 fixed = 271 total (was 262) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 57s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14701 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879732/HADOOP-14701.2.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 9dae0feb2d0a 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2be9412 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12906/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12906/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12906/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12906/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Configuration can log misleading warnings about an attempt to override final
> parameter
> --
>
> Key:
[jira] [Commented] (HADOOP-14644) Increase max heap size of Maven javadoc plugin
[
https://issues.apache.org/jira/browse/HADOOP-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108124#comment-16108124
]
Hudson commented on HADOOP-14644:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12083 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12083/])
HADOOP-14644. Increase max heap size of Maven javadoc plugin. (wang: rev
2be9412b73ae4308c5cee0186520fc2ad6d54e43)
* (edit) hadoop-project-dist/pom.xml
> Increase max heap size of Maven javadoc plugin
> --
>
> Key: HADOOP-14644
> URL: https://issues.apache.org/jira/browse/HADOOP-14644
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14644.01.patch
>
>
> It's easy to reproduce:
> {code}cd hadoop-common-project/hadoop-common/
> mvn javadoc:test-javadoc{code}
> The build will fail. I tried it on OSX (2 different ones).
> {code}ERROR] javadoc: error - Java heap space
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/jre/../bin/javadoc
> -J-Xmx512m @options @packages{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14343) Wrong pid file name in error message when starting secure daemon
[
https://issues.apache.org/jira/browse/HADOOP-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108119#comment-16108119
]
Hadoop QA commented on HADOOP-14343:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s{color} | {color:green} The patch generated 0 new + 20 unchanged - 1 fixed =
20 total (was 21) {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
10s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
3s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14343 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/1283/HADOOP-14343.02.patch
|
| Optional Tests | asflicense mvnsite unit shellcheck shelldocs |
| uname | Linux 75a263d88d6d 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 2be9412 |
| shellcheck | v0.4.6 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12905/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12905/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Wrong pid file name in error message when starting secure daemon
>
>
> Key: HADOOP-14343
> URL: https://issues.apache.org/jira/browse/HADOOP-14343
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-14343.01.patch, HADOOP-14343.02.patch
>
>
> {code}# this is for the daemon pid creation
> #shellcheck disable=SC2086
> echo $! > "${jsvcpidfile}" 2>/dev/null
> if [[ $? -gt 0 ]]; then
> hadoop_error "ERROR: Cannot write ${daemonname} pid ${daemonpidfile}."
> fi{code}
> It will log datanode's pid file instead of JSVC's pid file.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108092#comment-16108092
]
xiangguang zheng commented on HADOOP-14676:
---
[~shv] Could you take a look when you have a chance?
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105846#comment-16105846
]
xiangguang zheng edited comment on HADOOP-14676 at 7/31/17 10:25 PM:
-
The manual steps I did to verify:
1. compile and run the unit test towards hadoop common module. When I run the
test, there were 9 test failure and 1 test error. But I have verified that
those failure and errors existed in hadoop trunk before I made any changes, and
none of test failure/error was related to this patch
2. Run local debugger on MiniHadoopClusterManager Class to test if DF load the
correct default interval value(fs.df.interval was removed from core-default.xml
when testing so that DF constructor was forced to read value from Constant
CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT instead of from
core-default.xml
was (Author: zhengxg3):
The manual steps I did to verify:
1. compile and run the unit test towards hadoop common module. When I run the
test, there were 9 test failure and 1 test error. But I have verified that
those failure and errors existed in hadoop trunk before I made any changes, and
none of test failure/error was related to this patch
2. Run local debugger in on MiniHadoopClusterManager Class to test if DF load
the correct default interval value(fs.df.interval was removed from
core-default.xml when testing so that DF constructor was forced to read value
from Constant CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT instead of
from core-default.xml
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[ https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HADOOP-14701: Attachment: HADOOP-14701.2.patch > Configuration can log misleading warnings about an attempt to override final > parameter > -- > > Key: HADOOP-14701 > URL: https://issues.apache.org/jira/browse/HADOOP-14701 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Reporter: Andrew Sherman > Attachments: HADOOP-14701.1.patch, HADOOP-14701.2.patch > > > The class Configuration (in hadoop-common-project) is used to hold > configuration properties. > The method loadResources() is used to load properties from a list of default > resources into a parameter called “properties” (which happens to always be > the field called “properties”). > This is done by calling the method loadResource() which is used to load > properties from a single resource into a parameter called “properties” > (which happens to always be the field called “properties”). > This is done by calling loadProperty() which is used to load a single > property into a parameter called “properties” (which is sometimes, but not > always, the field called “properties”). > In loadProperty() there is some logic for dealing with properties that are > marked as final. > (1) a final property is not set into the “properties” parameter. > (2) a warning is printed for a final property if the value of the property > with the same name in the “properties” parameter is not equal to the new > value, and the property name is in the finalParameters field. > When loadResource() is reading properties from a stream it creates a > temporary Properties object that it populates by calling loadProperty(). This > is the case where loadProperty() has a parameter called “properties” which > is not the same as the field called “properties”). > When loadProperty() is called on a final property then because of (1) it is > not set into the “properties” parameter. But sometimes the property name is > already in the finalParameters field and now (2) the warning is triggered > because the property is not present in the “properties” parameter (although > it IS present in the “properties” field). The warning is misleading and > confusing. > h3. Possible Fix > One way of looking at the problem is that loadProperty is using the > finalParameters field in combination with the “properties” parameter. Change > the code to only generate a warning for a final property if the value of the > property with the same name in the “properties” field is not equal to the new > value, and the property name is in the finalParameters field. > I’ve made a patch containing this tiny change. > Note that I have done very minimal testing of this fix > h3. Example warning: > 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration: > [HiveServer2-Background-Pool: Thread-77]: > hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14676) Wrong default value for "fs.df.interval"
[
https://issues.apache.org/jira/browse/HADOOP-14676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105846#comment-16105846
]
xiangguang zheng edited comment on HADOOP-14676 at 7/31/17 10:23 PM:
-
The manual steps I did to verify:
1. compile and run the unit test towards hadoop common module. When I run the
test, there were 9 test failure and 1 test error. But I have verified that
those failure and errors existed in hadoop trunk before I made any changes, and
none of test failure/error was related to this patch
2. Run local debugger in on MiniHadoopClusterManager Class to test if DF load
the correct default interval value(fs.df.interval was removed from
core-default.xml when testing so that DF constructor was forced to read value
from Constant CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT instead of
from core-default.xml
was (Author: zhengxg3):
The manual steps I did to verify is to compile and run the unit test towards
hadoop common module. When I run the test, there were 9 test failure and 1 test
error. But I have verified that those failure and errors existed in hadoop
trunk before I made any changes, and none of test failure/error was related to
this patch
> Wrong default value for "fs.df.interval"
>
>
> Key: HADOOP-14676
> URL: https://issues.apache.org/jira/browse/HADOOP-14676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
>Affects Versions: 2.6.1
>Reporter: Konstantin Shvachko
>Assignee: xiangguang zheng
> Attachments: HADOOP-14676.001.patch
>
>
> According to {{core-default.xml}} the default value of {{fs.df.interval = 60
> sec}}. But the implementation of {{DF}} substitutes 3 sec instead. The
> problem is that {{DF}} uses outdated constant {{DF.DF_INTERVAL_DEFAULT}}
> instead of the correct one
> {{CommonConfigurationKeysPublic.FS_DF_INTERVAL_DEFAULT}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14343) Wrong pid file name in error message when starting secure daemon
[
https://issues.apache.org/jira/browse/HADOOP-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108082#comment-16108082
]
Eric Payne commented on HADOOP-14343:
-
[~boky01], thanks for the effort on this patch.
Patch LGTM. +1
[~aw], did you have anything you wanted to add?
> Wrong pid file name in error message when starting secure daemon
>
>
> Key: HADOOP-14343
> URL: https://issues.apache.org/jira/browse/HADOOP-14343
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-14343.01.patch, HADOOP-14343.02.patch
>
>
> {code}# this is for the daemon pid creation
> #shellcheck disable=SC2086
> echo $! > "${jsvcpidfile}" 2>/dev/null
> if [[ $? -gt 0 ]]; then
> hadoop_error "ERROR: Cannot write ${daemonname} pid ${daemonpidfile}."
> fi{code}
> It will log datanode's pid file instead of JSVC's pid file.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108066#comment-16108066
]
Hadoop QA commented on HADOOP-14397:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
15s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
35s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
4s{color} | {color:green} root: The patch generated 0 new + 139 unchanged - 2
fixed = 139 total (was 141) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
4s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
26s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14397 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879705/HADOOP-14397.004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3d8e3964d07a 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3e23415 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12903/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12903/testReport/ |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs-client U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12903/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT
[jira] [Updated] (HADOOP-13707) If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed
[
https://issues.apache.org/jira/browse/HADOOP-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated HADOOP-13707:
-
Target Version/s: 3.0.0-alpha2, 2.8.0, 2.9.0, 2.7.5 (was: 2.8.0, 2.9.0,
3.0.0-alpha2)
> If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot
> be accessed
> -
>
> Key: HADOOP-13707
> URL: https://issues.apache.org/jira/browse/HADOOP-13707
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Yuanbo Liu
>Assignee: Yuanbo Liu
> Labels: security
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HADOOP-13707.001.patch, HADOOP-13707.002.patch,
> HADOOP-13707.003.patch, HADOOP-13707.004.patch,
> HADOOP-13707-branch-2.8.patch, HADOOP-13707-branch-2-addendum.patch,
> HADOOP-13707-branch-2.patch
>
>
> In {{HttpServer2#hasAdministratorAccess}}, it uses
> `hadoop.security.authorization` to detect whether HTTP is authenticated.
> It's not correct, because enabling Kerberos and HTTP SPNEGO are two steps. If
> Kerberos is enabled while HTTP SPNEGO is not, some links cannot be accessed,
> such as "/logs", and it will return error message as below:
> {quote}
> HTTP ERROR 403
> Problem accessing /logs/. Reason:
> User dr.who is unauthorized to access this page.
> {quote}
> We should make sure {{HttpServletRequest#getAuthType}} is not null before we
> invoke {{HttpServer2#hasAdministratorAccess}}.
> {{getAuthType}} means to get the authorization scheme of this request
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14644) Increase max heap size of Maven javadoc plugin
[
https://issues.apache.org/jira/browse/HADOOP-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-14644:
-
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
2.9.0
Status: Resolved (was: Patch Available)
Thanks for the contribution Andras! Committed to trunk and branch-2.
> Increase max heap size of Maven javadoc plugin
> --
>
> Key: HADOOP-14644
> URL: https://issues.apache.org/jira/browse/HADOOP-14644
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HADOOP-14644.01.patch
>
>
> It's easy to reproduce:
> {code}cd hadoop-common-project/hadoop-common/
> mvn javadoc:test-javadoc{code}
> The build will fail. I tried it on OSX (2 different ones).
> {code}ERROR] javadoc: error - Java heap space
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/jre/../bin/javadoc
> -J-Xmx512m @options @packages{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14644) Increase max heap size of Maven javadoc plugin
[
https://issues.apache.org/jira/browse/HADOOP-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-14644:
-
Summary: Increase max heap size of Maven javadoc plugin (was:
javadoc:test-javadoc fails with OutOfMemoryError)
> Increase max heap size of Maven javadoc plugin
> --
>
> Key: HADOOP-14644
> URL: https://issues.apache.org/jira/browse/HADOOP-14644
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14644.01.patch
>
>
> It's easy to reproduce:
> {code}cd hadoop-common-project/hadoop-common/
> mvn javadoc:test-javadoc{code}
> The build will fail. I tried it on OSX (2 different ones).
> {code}ERROR] javadoc: error - Java heap space
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/jre/../bin/javadoc
> -J-Xmx512m @options @packages{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14644) javadoc:test-javadoc fails with OutOfMemoryError
[
https://issues.apache.org/jira/browse/HADOOP-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108059#comment-16108059
]
Andrew Wang commented on HADOOP-14644:
--
Seems innocuous enough to me. Let's get this in, thanks for contributing this
Andras!
> javadoc:test-javadoc fails with OutOfMemoryError
>
>
> Key: HADOOP-14644
> URL: https://issues.apache.org/jira/browse/HADOOP-14644
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14644.01.patch
>
>
> It's easy to reproduce:
> {code}cd hadoop-common-project/hadoop-common/
> mvn javadoc:test-javadoc{code}
> The build will fail. I tried it on OSX (2 different ones).
> {code}ERROR] javadoc: error - Java heap space
> [ERROR]
> [ERROR] Command line was:
> /Library/Java/JavaVirtualMachines/jdk1.8.0_74.jdk/Contents/Home/jre/../bin/javadoc
> -J-Xmx512m @options @packages{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-8621) FileUtil.symLink fails if spaces in path
[
https://issues.apache.org/jira/browse/HADOOP-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108060#comment-16108060
]
Hadoop QA commented on HADOOP-8621:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hadoop's
naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute
for instructions. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HADOOP-8621 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-8621 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12537980/hadoop-8621.txt |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12904/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FileUtil.symLink fails if spaces in path
>
>
> Key: HADOOP-8621
> URL: https://issues.apache.org/jira/browse/HADOOP-8621
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Robert Fuller
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: hadoop-8621.txt, patch.txt
>
>
> the 'ln -s' command fails in the current implementation if there is a space
> in the path for the target or linkname. A small change resolves the issue.
> String cmd = "ln -s " + target + " " + linkname;
> //Process p = Runtime.getRuntime().exec(cmd, null); //broken
> Process p = Runtime.getRuntime().exec(new
> String[]{"ln","-s",target,linkname}, null);
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14420) generateReports property is not applicable for maven-site-plugin:attach-descriptor goal
[
https://issues.apache.org/jira/browse/HADOOP-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-14420:
-
Resolution: Fixed
Fix Version/s: 3.0.0-beta1
Status: Resolved (was: Patch Available)
Thank you for the contribution Andras!
> generateReports property is not applicable for
> maven-site-plugin:attach-descriptor goal
> ---
>
> Key: HADOOP-14420
> URL: https://issues.apache.org/jira/browse/HADOOP-14420
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Fix For: 3.0.0-beta1
>
> Attachments: HADOOP-14420.01.patch
>
>
> In Hadoop Main's pom.xml:
> {code}
> maven-site-plugin
>
>
> attach-descriptor
>
> attach-descriptor
>
>
> true
>
>
>
> {code}
> [generateReports property is an invalid property for this
> goal|https://maven.apache.org/plugins/maven-site-plugin/attach-descriptor-mojo.html].
> It's only valid for [site
> goal|https://maven.apache.org/plugins/maven-site-plugin/site-mojo.html].
> Anyway, the default value is true so it can be removed.
> One less red line in IDEA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-8621) FileUtil.symLink fails if spaces in path
[
https://issues.apache.org/jira/browse/HADOOP-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HADOOP-8621:
-
Status: Patch Available (was: Open)
> FileUtil.symLink fails if spaces in path
>
>
> Key: HADOOP-8621
> URL: https://issues.apache.org/jira/browse/HADOOP-8621
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Robert Fuller
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: hadoop-8621.txt, patch.txt
>
>
> the 'ln -s' command fails in the current implementation if there is a space
> in the path for the target or linkname. A small change resolves the issue.
> String cmd = "ln -s " + target + " " + linkname;
> //Process p = Runtime.getRuntime().exec(cmd, null); //broken
> Process p = Runtime.getRuntime().exec(new
> String[]{"ln","-s",target,linkname}, null);
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14420) generateReports property is not applicable for maven-site-plugin:attach-descriptor goal
[
https://issues.apache.org/jira/browse/HADOOP-14420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108053#comment-16108053
]
Andrew Wang commented on HADOOP-14420:
--
+1 LGTM will commit shortly
> generateReports property is not applicable for
> maven-site-plugin:attach-descriptor goal
> ---
>
> Key: HADOOP-14420
> URL: https://issues.apache.org/jira/browse/HADOOP-14420
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-14420.01.patch
>
>
> In Hadoop Main's pom.xml:
> {code}
> maven-site-plugin
>
>
> attach-descriptor
>
> attach-descriptor
>
>
> true
>
>
>
> {code}
> [generateReports property is an invalid property for this
> goal|https://maven.apache.org/plugins/maven-site-plugin/attach-descriptor-mojo.html].
> It's only valid for [site
> goal|https://maven.apache.org/plugins/maven-site-plugin/site-mojo.html].
> Anyway, the default value is true so it can be removed.
> One less red line in IDEA.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-11790) leveldb usage should be disabled by default or smarter about platforms
[ https://issues.apache.org/jira/browse/HADOOP-11790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108041#comment-16108041 ] Andrew Wang commented on HADOOP-11790: -- Can this be targeted at a future 3.x release, or does it need to be in beta? What is our best guess at how long it would take to migrate to RocksDB? My guess is that this is backwards compatible and would be more than 1.5 months of work (how far we are from the mid-Sept target release date), so I don't want to hold the beta1 release for this. I can add a 3.0.0 fix version if you'd like to retarget this at GA, or we can lower the priority. > leveldb usage should be disabled by default or smarter about platforms > -- > > Key: HADOOP-11790 > URL: https://issues.apache.org/jira/browse/HADOOP-11790 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha3 > Environment: * any non-x86 > * any OS that isn't Linux, OSX, Windows >Reporter: Ayappan >Priority: Blocker > > The leveldbjni artifact in maven repository has been built for only x86 > architecture. Due to which some of the testcases are failing in PowerPC. The > leveldbjni community has no plans to support other platforms [ > https://github.com/fusesource/leveldbjni/issues/54 ]. Right now , the > approach is we need to locally built leveldbjni prior to running hadoop > testcases. Pushing a PowerPC-specific leveldbjni artifact in central maven > repository and making pom.xml to pickup it up while running in PowerPC is > another option but i don't know whether this is a suitable one . Any other > alternative/solution is there ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108031#comment-16108031 ] Andrew Wang commented on HADOOP-14693: -- Bold JIRA :) We haven't even finished our migration from JUnit 3 to 4. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14495) Add set options interface to FSDataOutputStreamBuilder
[
https://issues.apache.org/jira/browse/HADOOP-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107941#comment-16107941
]
Manoj Govindassamy commented on HADOOP-14495:
-
Looks overall good to me. Thanks for working on this [~eddyxu]. Few questions
below
{{TestLocalFileSystem}}
>> line 753, new BuilderWithSupportedKeys(unsupportedKeys, fileSys, path);
1. The variable naming is little confusing here - unsupportdKeys is passed to
the BuilderWithSupportedKeys ?
2. Can we please extend this test to run #opt() and #must() on the same key,
like "strM", just to cover the corner cases ?
{{FSDataOutputStreamBuilder}},
line 51, the comment can say #opt(key, value) and #must(key, value) to be more
generic.
line 303, like other interfaces, @Nonnull annotation may be needed for the
String value
> Add set options interface to FSDataOutputStreamBuilder
> ---
>
> Key: HADOOP-14495
> URL: https://issues.apache.org/jira/browse/HADOOP-14495
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs
>Affects Versions: 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14495.00.patch, HADOOP-14495.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[
https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107919#comment-16107919
]
Hadoop QA commented on HADOOP-14701:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
4s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14701 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879450/HADOOP-14701.1.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3f1974234343 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3e23415 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12902/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12902/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Configuration can log misleading warnings about an attempt to override final
> parameter
> --
>
> Key: HADOOP-14701
> URL: https://issues.apache.org/jira/browse/HADOOP-14701
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
>Reporter: Andrew Sherman
> Attachments: HADOOP-14701.1.patch
>
>
> The class Configuration (in hadoop-common-project) is used to hold
>
[jira] [Updated] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HADOOP-14397:
---
Attachment: HADOOP-14397.004.patch
Fixed {{TestLocalFileSystem}} in patch 04. TestZKFailoverController and
TestKDiag passed on my laptop.
[~manojg], could you take another look?
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch, HADOOP-14397.004.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14703) ConsoleSink for metrics2
[
https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107887#comment-16107887
]
Hadoop QA commented on HADOOP-14703:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
34s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
53s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 53s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 22 new + 0 unchanged - 0 fixed = 22 total (was 0) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
35s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 39s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14703 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879658/0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 138d1fcc3cb5 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 3e23415 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/patch-compile-root.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/patch-compile-root.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12901/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
|
| unit |
[jira] [Commented] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[
https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107855#comment-16107855
]
Hadoop QA commented on HADOOP-14245:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
53s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
7s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 58s{color} | {color:orange} root: The patch generated 7 new + 17 unchanged -
7 fixed = 24 total (was 24) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 27s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
26s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14245 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879601/HADOOP-14245.02.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 836ab3edf222 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd6d0f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
[jira] [Commented] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107818#comment-16107818 ] John Zhuge commented on HADOOP-14703: - [~macmaster] Very useful contribution! When you name the patch file, could you please follow the naming convention in https://wiki.apache.org/hadoop/HowToContribute#Naming_your_patch? > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-10758) KMS: add ACLs on per key basis.
[ https://issues.apache.org/jira/browse/HADOOP-10758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107792#comment-16107792 ] Xiao Chen commented on HADOOP-10758: Thanks for the continuing on this [~lars_francke], and for the interest for a doc patch. I agree the cluster may have security holes exposed, if the ACLs are not configured correctly. The solution IMO, is to make our docs better so people are aware of this, and more unlikely to configure their cluster to expose such holes. I don't see a way of software level 'fix' to stop mis-configurations. Perhaps we can log a warning if MANAGEMENT is open to everyone, to be proactive. [HDFS permission guide|http://hadoop.apache.org/docs/r3.0.0-alpha2/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html] may also come in handy for some ideas to improve this. > KMS: add ACLs on per key basis. > --- > > Key: HADOOP-10758 > URL: https://issues.apache.org/jira/browse/HADOOP-10758 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 3.0.0-alpha1 >Reporter: Alejandro Abdelnur >Assignee: Arun Suresh > Fix For: 2.6.0 > > Attachments: HADOOP-10758.1.patch, HADOOP-10758.2.patch, > HADOOP-10758.3.patch, HADOOP-10758.4.patch, HADOOP-10758.5.patch, > HADOOP-10758.6.patch, HADOOP-10758.7.patch, HADOOP-10758.8.patch, > HADOOP-10758.9.patch > > > The KMS server should enforce ACLs on per key basis. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14667) Flexible Visual Studio support
[ https://issues.apache.org/jira/browse/HADOOP-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107779#comment-16107779 ] Chris Douglas commented on HADOOP-14667: bq. with a bit more work, I'm fairly confident I can get an "on the fly" upgrade-in-place Soright. If it's easier to leave this for future work, don't get hung up on it. LMK when this is ready to test, I'll try a fresh install of VS2017 community edition. > Flexible Visual Studio support > -- > > Key: HADOOP-14667 > URL: https://issues.apache.org/jira/browse/HADOOP-14667 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-beta1 > Environment: Windows >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-14667.00.patch, HADOOP-14667.01.patch, > HADOOP-14667.02.patch, HADOOP-14667.03.patch, HADOOP-14667.04.patch > > > Is it time to upgrade the Windows native project files to use something more > modern than Visual Studio 2010? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14667) Flexible Visual Studio support
[ https://issues.apache.org/jira/browse/HADOOP-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107724#comment-16107724 ] Allen Wittenauer edited comment on HADOOP-14667 at 7/31/17 6:33 PM: bq. Does this mean the build will modify files that committers have to remember to unstage before commit? That's exactly what that means. devenv upgrades the solution (sln) and VS project (vxcproj) files for the new version by overwriting them. From what I can tell, it's usually just updating the XML schema version and setting some version flags. (It's sort of ridiculous really, but whatever.) There's also the backup dirs and logs it creates. All of those changes stick out when doing a git diff or a git status. But yeah, with a bit more work, I'm fairly confident I can get an "on the fly" upgrade-in-place. Only requirement would be that vcvarsall be run one time prior to doing any maven work so that the paths are established. was (Author: aw): bq. Does this mean the build will modify files that committers have to remember to unstage before commit? That's exactly what that means. devenv upgrades the solution (sln) and VS project (vxcproj) files for the new version. From what I can tell, it's usually just updating the XML schema version and setting some version flags. (It's sort of ridiculous really, but whatever.) There's also the backup dirs and logs it creates. All of those changes stick out when doing a git diff or a git status. But yeah, with a bit more work, I'm fairly confident I can get an "on the fly" upgrade-in-place. Only requirement would be that vcvarsall be run one time prior to doing any maven work so that the paths are established. > Flexible Visual Studio support > -- > > Key: HADOOP-14667 > URL: https://issues.apache.org/jira/browse/HADOOP-14667 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-beta1 > Environment: Windows >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-14667.00.patch, HADOOP-14667.01.patch, > HADOOP-14667.02.patch, HADOOP-14667.03.patch, HADOOP-14667.04.patch > > > Is it time to upgrade the Windows native project files to use something more > modern than Visual Studio 2010? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14667) Flexible Visual Studio support
[ https://issues.apache.org/jira/browse/HADOOP-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107724#comment-16107724 ] Allen Wittenauer commented on HADOOP-14667: --- bq. Does this mean the build will modify files that committers have to remember to unstage before commit? That's exactly what that means. devenv upgrades the solution (sln) and VS project (vxcproj) files for the new version. From what I can tell, it's usually just updating the XML schema version and setting some version flags. (It's sort of ridiculous really, but whatever.) There's also the backup dirs and logs it creates. All of those changes stick out when doing a git diff or a git status. But yeah, with a bit more work, I'm fairly confident I can get an "on the fly" upgrade-in-place. Only requirement would be that vcvarsall be run one time prior to doing any maven work so that the paths are established. > Flexible Visual Studio support > -- > > Key: HADOOP-14667 > URL: https://issues.apache.org/jira/browse/HADOOP-14667 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-beta1 > Environment: Windows >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-14667.00.patch, HADOOP-14667.01.patch, > HADOOP-14667.02.patch, HADOOP-14667.03.patch, HADOOP-14667.04.patch > > > Is it time to upgrade the Windows native project files to use something more > modern than Visual Studio 2010? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14667) Flexible Visual Studio support
[ https://issues.apache.org/jira/browse/HADOOP-14667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107702#comment-16107702 ] Chris Douglas commented on HADOOP-14667: bq. As long as committers don't try to commit upgraded files, this might be ok. ? Does this mean the build will modify files that committers have to remember to unstage before commit? Even if that's the case, very few committers push from Windows. I'd trade that for more modern VS support. > Flexible Visual Studio support > -- > > Key: HADOOP-14667 > URL: https://issues.apache.org/jira/browse/HADOOP-14667 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-beta1 > Environment: Windows >Reporter: Allen Wittenauer >Assignee: Allen Wittenauer > Attachments: HADOOP-14667.00.patch, HADOOP-14667.01.patch, > HADOOP-14667.02.patch, HADOOP-14667.03.patch, HADOOP-14667.04.patch > > > Is it time to upgrade the Windows native project files to use something more > modern than Visual Studio 2010? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14701) Configuration can log misleading warnings about an attempt to override final parameter
[ https://issues.apache.org/jira/browse/HADOOP-14701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HADOOP-14701: Status: Patch Available (was: Open) > Configuration can log misleading warnings about an attempt to override final > parameter > -- > > Key: HADOOP-14701 > URL: https://issues.apache.org/jira/browse/HADOOP-14701 > Project: Hadoop Common > Issue Type: Bug > Components: conf >Reporter: Andrew Sherman > Attachments: HADOOP-14701.1.patch > > > The class Configuration (in hadoop-common-project) is used to hold > configuration properties. > The method loadResources() is used to load properties from a list of default > resources into a parameter called “properties” (which happens to always be > the field called “properties”). > This is done by calling the method loadResource() which is used to load > properties from a single resource into a parameter called “properties” > (which happens to always be the field called “properties”). > This is done by calling loadProperty() which is used to load a single > property into a parameter called “properties” (which is sometimes, but not > always, the field called “properties”). > In loadProperty() there is some logic for dealing with properties that are > marked as final. > (1) a final property is not set into the “properties” parameter. > (2) a warning is printed for a final property if the value of the property > with the same name in the “properties” parameter is not equal to the new > value, and the property name is in the finalParameters field. > When loadResource() is reading properties from a stream it creates a > temporary Properties object that it populates by calling loadProperty(). This > is the case where loadProperty() has a parameter called “properties” which > is not the same as the field called “properties”). > When loadProperty() is called on a final property then because of (1) it is > not set into the “properties” parameter. But sometimes the property name is > already in the finalParameters field and now (2) the warning is triggered > because the property is not present in the “properties” parameter (although > it IS present in the “properties” field). The warning is misleading and > confusing. > h3. Possible Fix > One way of looking at the problem is that loadProperty is using the > finalParameters field in combination with the “properties” parameter. Change > the code to only generate a warning for a final property if the value of the > property with the same name in the “properties” field is not equal to the new > value, and the property name is in the finalParameters field. > I’ve made a patch containing this tiny change. > Note that I have done very minimal testing of this fix > h3. Example warning: > 2017-07-25 12:14:24,639 WARN org.apache.hadoop.conf.Configuration: > [HiveServer2-Background-Pool: Thread-77]: > hdfs://asherman-cdh6-subset-parcel-1.gce.cloudera.com:8020/user/hive/.staging/job_1501008923167_0001/job.xml:an > attempt to override final parameter: > mapreduce.job.end-notification.max.retry.interval; Ignoring. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[
https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107668#comment-16107668
]
Andras Bokor commented on HADOOP-14698:
---
Thanks [~msingh] for letting me know. What do you think the correct resolution
is?
* We can wait in {{postProcessPath}} until the executor finishes.
* or we can disallow using -t in case of moveFromLocal
* other.
> Make copyFromLocal's -t option available for put as well
>
>
> Key: HADOOP-14698
> URL: https://issues.apache.org/jira/browse/HADOOP-14698
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14698.01.patch, HADOOP-14698.02.patch
>
>
> After HDFS-11786 copyFromLocal and put are no longer identical.
> I do not see any reason why not to add the new feature to put as well.
> Being non-identical makes the understanding/usage of command more complicated
> from user point of view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14397) Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it
[
https://issues.apache.org/jira/browse/HADOOP-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107665#comment-16107665
]
Lei (Eddy) Xu commented on HADOOP-14397:
{{TestLocalFileSystem}} is related. Working on it.
> Pull up the builder pattern to FileSystem and add AbstractContractCreateTest
> for it
> ---
>
> Key: HADOOP-14397
> URL: https://issues.apache.org/jira/browse/HADOOP-14397
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, fs, hdfs-client
>Affects Versions: 2.9.0, 3.0.0-alpha3
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HADOOP-14397.000.patch, HADOOP-14397.001.patch,
> HADOOP-14397.002.patch, HADOOP-14397.003.patch
>
>
> After reach the stability of the Builder APIs, we should promote the API from
> {{DistributedFileSystem}} to {{FileSystem}}, and add necessary contract tests
> to cover the API for all file systems.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107633#comment-16107633 ] Ronald Macmaster commented on HADOOP-14703: --- That should be a simple fix. I'd just have to properly translate the uses of System.out to LOG. Should we call it an Slf4jSink? LogSink? > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14703: -- Status: Patch Available (was: Open) > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107623#comment-16107623 ] Allen Wittenauer commented on HADOOP-14703: --- Rather use stdio, it'd be nice to use slf4j here. There are a lot of advantages: * users can send to console or to a file via log4j.properties or something fancier with their own slf4j plug-in * effectively could EOL FileSink * could replace that awful hack that's in HDFS to also send metrics to a logfile > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Macmaster updated HADOOP-14703: -- Tags: hadoop-common metrics metrics2 debugging (was: hadoop-common metrics2 debugging) > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14704: -- Release Note: (was: hadoop-tools/hadoop-kafka already contains a KafkaSink.) > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107615#comment-16107615 ] Allen Wittenauer commented on HADOOP-14704: --- Yeah, the new plan is for all sinks that have external dependencies to go into hadoop-tools so that they don't inject their dependencies into mainline. :) > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14704: -- Fix Version/s: (was: 3.0.0-beta1) > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Macmaster resolved HADOOP-14704. --- Resolution: Duplicate Fix Version/s: 3.0.0-beta1 Release Note: hadoop-tools/hadoop-kafka already contains a KafkaSink. > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Fix For: 3.0.0-beta1 > > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer reassigned HADOOP-14703: - Assignee: Ronald Macmaster > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster >Assignee: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14703: -- Fix Version/s: (was: 3.0.0-beta1) > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14703: -- Component/s: metrics > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Labels: newbie > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107613#comment-16107613 ] Ronald Macmaster commented on HADOOP-14704: --- Oh, nice. I didn't see that there. > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14704: -- Component/s: metrics > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common, metrics >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107603#comment-16107603 ] Allen Wittenauer commented on HADOOP-14704: --- You've seen hadoop-tools/hadoop-kafka, right? > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HADOOP-14704: -- Fix Version/s: (was: 3.0.0-beta1) > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107594#comment-16107594 ] Andras Bokor edited comment on HADOOP-14245 at 7/31/17 5:02 PM: Thanks, I do not know how could I miss it... was (Author: boky01): Thank, I do not know how could I miss it... > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107594#comment-16107594 ] Andras Bokor commented on HADOOP-14245: --- Thank, I do not know how could I miss it... > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-14245: -- Status: Patch Available (was: Open) > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107554#comment-16107554 ] Akira Ajisaka commented on HADOOP-14245: You need to hit "submit patch" button to set the status to "patch available". > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14704) KafkaSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Macmaster updated HADOOP-14704: -- Description: The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. The regular kafka configuration parameters can be directly specified in the hadoop-metrics2.properties file under the metrics2 sink prefix (ex. test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). Addition of the KafkaSink will enrich the metrics2 sink repertory that already includes sinks for Graphite and Ganglia. was: The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. The regular kafka configuration parameters can be directly specified in the hadoop-metrics2.properties file under the metrics2 sink prefix (ex. test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > KafkaSink for metrics2 > -- > > Key: HADOOP-14704 > URL: https://issues.apache.org/jira/browse/HADOOP-14704 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Fix For: 3.0.0-beta1 > > > The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. > The regular kafka configuration parameters can be directly specified in the > hadoop-metrics2.properties file under the metrics2 sink prefix (ex. > test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). > Addition of the KafkaSink will enrich the metrics2 sink repertory that > already includes sinks for Graphite and Ganglia. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14698) Make copyFromLocal's -t option available for put as well
[
https://issues.apache.org/jira/browse/HADOOP-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107552#comment-16107552
]
Mukul Kumar Singh commented on HADOOP-14698:
Thanks for the patch [~boky01], the patch looks good however please take a look
at the MoveFromLocal.
Currently multi threading only works for files (i.e. leaf nodes of the tree)
while directory traversing is still single threaded.
If this code is changed, then delete of directory will be tried right after all
the files in the directory are enqueued as tasks.
This will cause the rmdir executed as part of post process to fail because
files are still present in the directory.
{code}
@Override
protected void postProcessPath(PathData src) throws IOException {
if (!src.fs.delete(src.path, false)) {
// we have no way to know the actual error...
PathIOException e = new PathIOException(src.toString());
e.setOperation("remove");
throw e;
}
}
{code}
> Make copyFromLocal's -t option available for put as well
>
>
> Key: HADOOP-14698
> URL: https://issues.apache.org/jira/browse/HADOOP-14698
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Andras Bokor
>Assignee: Andras Bokor
> Attachments: HADOOP-14698.01.patch, HADOOP-14698.02.patch
>
>
> After HDFS-11786 copyFromLocal and put are no longer identical.
> I do not see any reason why not to add the new feature to put as well.
> Being non-identical makes the understanding/usage of command more complicated
> from user point of view.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14704) KafkaSink for metrics2
Ronald Macmaster created HADOOP-14704: - Summary: KafkaSink for metrics2 Key: HADOOP-14704 URL: https://issues.apache.org/jira/browse/HADOOP-14704 Project: Hadoop Common Issue Type: Improvement Components: common Affects Versions: 3.0.0-beta1 Reporter: Ronald Macmaster Fix For: 3.0.0-beta1 The KafkaSink, acting as a KafkaProducer, publishes metrics to a Kafka topic. The regular kafka configuration parameters can be directly specified in the hadoop-metrics2.properties file under the metrics2 sink prefix (ex. test.sink.kafka.bootstrap.servers or client.sink.kafka.compression.type). -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Macmaster updated HADOOP-14703: -- Attachment: 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0-beta1 > > Attachments: > 0001-HADOOP-14703.-ConsoleSink-for-simple-metrics-printin.patch > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-9747) Reduce unnecessary UGI synchronization
[
https://issues.apache.org/jira/browse/HADOOP-9747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107521#comment-16107521
]
Hadoop QA commented on HADOOP-9747:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
17s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 37s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 9 new + 117 unchanged - 25 fixed = 126 total (was 142) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 19s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestKDiag |
| | hadoop.security.token.delegation.TestZKDelegationTokenSecretManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-9747 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12861100/HADOOP-9747.2.trunk.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 005ffdae540d 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd6d0f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12899/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12899/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12899/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12899/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Reduce unnecessary UGI synchronization
> --
>
> Key: HADOOP-9747
>
[jira] [Created] (HADOOP-14703) ConsoleSink for metrics2
Ronald Macmaster created HADOOP-14703: - Summary: ConsoleSink for metrics2 Key: HADOOP-14703 URL: https://issues.apache.org/jira/browse/HADOOP-14703 Project: Hadoop Common Issue Type: Improvement Components: common Affects Versions: 3.0.0-beta1 Reporter: Ronald Macmaster Fix For: 3.0.0-beta1 The ConsoleSink will provide a simple solution to dump metrics to the console through std.out. Quick access to metrics through the console will simplify the development, testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14703) ConsoleSink for metrics2
[ https://issues.apache.org/jira/browse/HADOOP-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Macmaster updated HADOOP-14703: -- Description: The ConsoleSink will provide a simple solution to dump metrics to the console through std.out. Quick access to metrics through the console will simplify the development, testing, and debugging process. was:The ConsoleSink will provide a simple solution to dump metrics to the console through std.out. Quick access to metrics through the console will simplify the development, testing, and debugging process. > ConsoleSink for metrics2 > > > Key: HADOOP-14703 > URL: https://issues.apache.org/jira/browse/HADOOP-14703 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-beta1 >Reporter: Ronald Macmaster > Labels: newbie > Fix For: 3.0.0-beta1 > > Original Estimate: 6h > Remaining Estimate: 6h > > The ConsoleSink will provide a simple solution to dump metrics to the console > through std.out. > Quick access to metrics through the console will simplify the development, > testing, and debugging process. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-3733) "s3:" URLs break when Secret Key contains a slash, even if encoded
[
https://issues.apache.org/jira/browse/HADOOP-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107425#comment-16107425
]
Vinayakumar B commented on HADOOP-3733:
---
Stripping down the userinfo part of provided uri in the filesystem's {{uri}}
makes the Distcp fail in following case.
1. AccessKey and Secret Key are provided in the URI itself (ideally it should
not, but for ease of use, old application uses old style uris)
2. While doing distcp of directory, {{fs.listStatus()}} results in paths
WITHOUT these AccessKeys and Secrets ( because fs.makeQualified() uses {{uri}}
without AK:SK)
3. Which eventually fails in Maps as there is no other way of credentials are
given.
As Release note says, supporting userinfo on URI itself would be useful for
distcp, but currently its breaking the existing functionality.
[~ste...@apache.org]/[~cnauroth], whether old functionality should be added
back?
> "s3:" URLs break when Secret Key contains a slash, even if encoded
> --
>
> Key: HADOOP-3733
> URL: https://issues.apache.org/jira/browse/HADOOP-3733
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 0.17.1, 2.0.2-alpha
>Reporter: Stuart Sierra
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 2.8.0, 3.0.0-alpha1
>
> Attachments: HADOOP-3733-20130223T011025Z.patch,
> HADOOP-3733-branch-2-001.patch, HADOOP-3733-branch-2-002.patch,
> HADOOP-3733-branch-2-003.patch, HADOOP-3733-branch-2-004.patch,
> HADOOP-3733-branch-2-005.patch, HADOOP-3733-branch-2-006.patch,
> HADOOP-3733-branch-2-007.patch, hadoop-3733.patch, HADOOP-3733.patch
>
>
> When using URLs of the form s3://ID:SECRET@BUCKET/ at the command line,
> distcp fails if the SECRET contains a slash, even when the slash is
> URL-encoded as %2F.
> Say your AWS Access Key ID is RYWX12N9WCY42XVOL8WH
> And your AWS Secret Key is Xqj1/NMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv
> And your bucket is called "mybucket"
> You can URL-encode the Secret KKey as
> Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv
> But this doesn't work:
> {noformat}
> $ bin/hadoop distcp file:///source
> s3://RYWX12N9WCY42XVOL8WH:Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv@mybucket/dest
> 08/07/09 15:05:22 INFO util.CopyFiles: srcPaths=[file:///source]
> 08/07/09 15:05:22 INFO util.CopyFiles:
> destPath=s3://RYWX12N9WCY42XVOL8WH:Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv@mybucket/dest
> 08/07/09 15:05:23 WARN httpclient.RestS3Service: Unable to access bucket:
> mybucket
> org.jets3t.service.S3ServiceException: S3 HEAD request failed.
> ResponseCode=403, ResponseMessage=Forbidden
> at
> org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:339)
> ...
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: org.apache.hadoop.fs.s3.S3Exception:
> org.jets3t.service.S3ServiceException: S3 PUT failed. XML Error Message:
> encoding="UTF-8"?>SignatureDoesNotMatchThe
> request signature we calculated does not match the signature you provided.
> Check your key and signing method.
> at
> org.apache.hadoop.fs.s3.Jets3tFileSystemStore.createBucket(Jets3tFileSystemStore.java:141)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[
https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107418#comment-16107418
]
Jeff Storck commented on HADOOP-14699:
--
[~daryn] Excellent! I've linked HADOOP-9747 to this JIRA.
> Impersonation errors with UGI after second principal relogin
>
>
> Key: HADOOP-14699
> URL: https://issues.apache.org/jira/browse/HADOOP-14699
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 2.6.2, 2.7.3, 2.8.1
>Reporter: Jeff Storck
>
> Multiple principals that are logged in using UGI instances that are
> instantiated from a UGI class loaded by the same classloader will encounter
> problems when the second principal attempts to relogin and perform an action
> using a UGI.doAs(). An impersonation will occur and the operation attempted
> by the second principal after relogging in will fail. There should not be an
> implicit attempt to impersonate the second principal through the first
> principal that logged in.
> I have created a GitHub project that exhibits the impersonation error with
> brief instructions on how to set up for the test and run it:
> https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
> {noformat}18:44:55.687 [pool-2-thread-2] WARN
> h.u.u.ugirunnable.ugite...@example.com - Unexpected exception while
> performing task for [ugite...@example.com (auth:KERBEROS)]
> org.apache.hadoop.ipc.RemoteException: User: ugite...@example.com is not
> allowed to impersonate ugite...@example.com
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1481)
> at org.apache.hadoop.ipc.Client.call(Client.java:1427)
> at org.apache.hadoop.ipc.Client.call(Client.java:1337)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1448)
> at
> hadoop.ugitest.UgiTestMain$UgiRunnable.lambda$run$2(UgiTestMain.java:194)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
> at hadoop.ugitest.UgiTestMain$UgiRunnable.run(UgiTestMain.java:194)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For
[jira] [Commented] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107410#comment-16107410 ] Andras Bokor commented on HADOOP-14245: --- Reattach the patch did not help. What else can I do? > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14699) Impersonation errors with UGI after second principal relogin
[
https://issues.apache.org/jira/browse/HADOOP-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107404#comment-16107404
]
Daryn Sharp commented on HADOOP-14699:
--
This will be fixed by HADOOP-9747 – if someone ever reviews it... Removing the
synchronization removed the class static fields that allow this to occur.
> Impersonation errors with UGI after second principal relogin
>
>
> Key: HADOOP-14699
> URL: https://issues.apache.org/jira/browse/HADOOP-14699
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 2.6.2, 2.7.3, 2.8.1
>Reporter: Jeff Storck
>
> Multiple principals that are logged in using UGI instances that are
> instantiated from a UGI class loaded by the same classloader will encounter
> problems when the second principal attempts to relogin and perform an action
> using a UGI.doAs(). An impersonation will occur and the operation attempted
> by the second principal after relogging in will fail. There should not be an
> implicit attempt to impersonate the second principal through the first
> principal that logged in.
> I have created a GitHub project that exhibits the impersonation error with
> brief instructions on how to set up for the test and run it:
> https://github.com/jtstorck/kerberos-examples/tree/master/hadoop/ugi-test
> {noformat}18:44:55.687 [pool-2-thread-2] WARN
> h.u.u.ugirunnable.ugite...@example.com - Unexpected exception while
> performing task for [ugite...@example.com (auth:KERBEROS)]
> org.apache.hadoop.ipc.RemoteException: User: ugite...@example.com is not
> allowed to impersonate ugite...@example.com
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1481)
> at org.apache.hadoop.ipc.Client.call(Client.java:1427)
> at org.apache.hadoop.ipc.Client.call(Client.java:1337)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
> at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1448)
> at
> hadoop.ugitest.UgiTestMain$UgiRunnable.lambda$run$2(UgiTestMain.java:194)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
> at hadoop.ugitest.UgiTestMain$UgiRunnable.run(UgiTestMain.java:194)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745){noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HADOOP-14672) Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, dom, etc.
[ https://issues.apache.org/jira/browse/HADOOP-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107363#comment-16107363 ] Arpit Agarwal commented on HADOOP-14672: Thanks for reviewing and committing this [~djp]. > Shaded Hadoop-client-minicluster include unshaded classes, like: javax, sax, > dom, etc. > -- > > Key: HADOOP-14672 > URL: https://issues.apache.org/jira/browse/HADOOP-14672 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Junping Du >Assignee: Bharat Viswanadham >Priority: Blocker > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-14672.02.patch, HADOOP-14672.03.patch, > HADOOP-14672.04.patch, HADOOP-14672.05.patch, HADOOP-14672.patch > > > The shaded hadoop-client-minicluster shouldn't include any unshaded > dependencies, but we can see: javax, dom, sax, etc. are all unshaded. > CC [~busbey] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14643) Clean up Test(HDFS|LocalFS)FileContextMainOperations and FileContextMainOperationsBaseTest
[ https://issues.apache.org/jira/browse/HADOOP-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107222#comment-16107222 ] Andras Bokor commented on HADOOP-14643: --- [~raviprak], Junits are unrelated. I cannot get +1 from Hadoop QA due to intermittent tests. > Clean up Test(HDFS|LocalFS)FileContextMainOperations and > FileContextMainOperationsBaseTest > -- > > Key: HADOOP-14643 > URL: https://issues.apache.org/jira/browse/HADOOP-14643 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14643.01.patch, HADOOP-14643.02.patch, > HADOOP-14643.03.patch > > > I was working with classes in summary. It's good time to clean up them as > "Boy Scout Rule" -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14643) Clean up Test(HDFS|LocalFS)FileContextMainOperations and FileContextMainOperationsBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107192#comment-16107192
]
Hadoop QA commented on HADOOP-14643:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 4m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
21s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
48s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
59s{color} | {color:green} root: The patch generated 0 new + 18 unchanged - 92
fixed = 18 total (was 110) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 18s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 53s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
46s{color} | {color:green} hadoop-azure-datalake in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}198m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14643 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879588/HADOOP-14643.03.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 5da33c16bb59 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 0fd6d0f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
[jira] [Updated] (HADOOP-14702) Site documentation should catch up with java sources
[ https://issues.apache.org/jira/browse/HADOOP-14702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Gu updated HADOOP-14702: -- Attachment: (was: HADOOP-14702-branch-2.001.patch) > Site documentation should catch up with java sources > > > Key: HADOOP-14702 > URL: https://issues.apache.org/jira/browse/HADOOP-14702 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.8.1, 3.0.0-alpha4 >Reporter: Doris Gu >Assignee: Doris Gu >Priority: Minor > Attachments: HADOOP-14702.001.patch, HADOOP-14702-branch-2.001.patch > > > There are some gaps between java sources and site documentations, 3.0.0 does > the work but still can do some improvements. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14702) Site documentation should catch up with java sources
[ https://issues.apache.org/jira/browse/HADOOP-14702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Gu updated HADOOP-14702: -- Attachment: HADOOP-14702-branch-2.001.patch > Site documentation should catch up with java sources > > > Key: HADOOP-14702 > URL: https://issues.apache.org/jira/browse/HADOOP-14702 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.8.1, 3.0.0-alpha4 >Reporter: Doris Gu >Assignee: Doris Gu >Priority: Minor > Attachments: HADOOP-14702.001.patch, HADOOP-14702-branch-2.001.patch > > > There are some gaps between java sources and site documentations, 3.0.0 does > the work but still can do some improvements. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14702) Site documentation should catch up with java sources
[ https://issues.apache.org/jira/browse/HADOOP-14702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doris Gu updated HADOOP-14702: -- Attachment: HADOOP-14702.001.patch > Site documentation should catch up with java sources > > > Key: HADOOP-14702 > URL: https://issues.apache.org/jira/browse/HADOOP-14702 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.8.1, 3.0.0-alpha4 >Reporter: Doris Gu >Assignee: Doris Gu >Priority: Minor > Attachments: HADOOP-14702.001.patch, HADOOP-14702-branch-2.001.patch > > > There are some gaps between java sources and site documentations, 3.0.0 does > the work but still can do some improvements. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107049#comment-16107049 ] Andras Bokor edited comment on HADOOP-14245 at 7/31/17 9:48 AM: Jenkins did not trigger pre-commit build for my patch. I have no right to start manually so I am attaching the same patch to kick [~hadoopqa]. was (Author: boky01): It seems Jenkins did not trigger pre-commit build for my patch. I have no right to start manually so I am attaching the same patch to kick [~hadoopqa]. > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14245) Use Mockito.when instead of Mockito.stub
[ https://issues.apache.org/jira/browse/HADOOP-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-14245: -- Attachment: HADOOP-14245.02.patch It seems Jenkins did not trigger pre-commit build for my patch. I have no right to start manually so I am attaching the same patch to kick [~hadoopqa]. > Use Mockito.when instead of Mockito.stub > > > Key: HADOOP-14245 > URL: https://issues.apache.org/jira/browse/HADOOP-14245 > Project: Hadoop Common > Issue Type: Test > Components: test >Reporter: Akira Ajisaka >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14245.01.patch, HADOOP-14245.02.patch > > > Mockito.stub was removed in Mockito 2. Mockito.when should be used instead. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
